Hacking Log

JingleFT in main branch

2012-04-12T17:57:00.001-07:00

Jingle file transfer has finally been merged into Gajim's default branch. Feel free to check it out, and report any bugs you find.

EDIT: Actually it is not yet merged right now, give it one week.

Stalled

2012-02-11T17:04:00.000-08:00

I'm still working on jingleFT for Gajim (not much progress there). The changes for xep-135 (no progress at all there). A secret project involving i2p (I don't talk about projects before having done a beta. No much progress there either).

The jingleFT branch of Gajim does need some refactoring. I made a state diagram of the file transfer, and it looks awful.

Clearly there are unnecessary states like transport info and proxy activated. The events are somewhat ambiguous, and the states accepted and initialized should be merged into one.

I'll fix it. See if I can shake off being so stalled.

Getting into I2P

2012-01-16T17:44:00.000-08:00

For a long time I have heard of I2P and I thought it was a good idea, but never actually tried to install it. Today I installed I2P and I have to say that I'm amazed by it. I was expecting a barely functioning app, very slow response times and crappy ported apps running on top of it. I was wrong.

The I2P router console is simply great. It include many apps, such as its own web, email and BitTorrent servers. I'm not very good at describing stuff, and reading about it's probably too dry. I would recommend anybody to try it first if they are interested. It is like discovering a new internet inside of the internet.

It is true that it lacks a lot of services, and that's why I want to know more about developing apps for I2P.

Jingle Bells, Jingle all the way

2011-12-31T18:56:00.000-08:00

What have I been doing? I used the holidays to finally get some work done in the jingle_FT branch of gajim. It's progressing just fine. I would hope that before my vacations ends it will be stable enough to be merged into the main branch.

It would be a really good help if you check out the branch and report some bugs.

It is easy to do just follow these steps:

1- Check out the latest jingleFT branch

hg clone http://hg.gajim.org/gajim gajim

cd gajim

hg update jingleFT

2- Report any bugs you find.

Go to https://trac.gajim.org/newticket

Proposed changes to XEP-0135

2011-11-27T17:47:00.000-08:00

Hi, I been working on some changes to XEP-0135.

* Replacing SI file transfer with jingle FT

* Replacing section 6, with a link pointing to section 5 of XEP-0234,
which already covers the same function.

* Adding support for pubsub, only for finding files using the method I
introduce bellow. It doesn't make much sense to traverse the directory
of every user subscribed to a pubsub, but it will make a lot of sense
searching for specific files. (XEP-0137 does not suffice for this)

For example:

A user is subscribed to the books pubsub channel. It sends a query,
looking for a book "Romeo and Juliet - By Shakespeare". The
subscribers reply if they get a match with information about their
files. The initiator requests the file from whoever has what he wants
and file transfer starts.

* A new section should be added to cover finding files by providing a
criteria, instead of just asking for all the files.

For example:

5.5 Finding Specific Files

Finding files by asking for a file list is not very practical if there
are too many files being shared. It is very resource intensive and it
is understood that the user may not be interested in all of the files,
but rather he or she would be interested in finding one specific file
or one specific kind of file (text, image or videos).

In order to do this, the identity stanza is used to match files by one
or more fields i.e. 'name', 'date', 'size', etc...

Example XX. Finding Specific Files


<iq type='get'

    from='hag66 at shakespeare.lit/pda'

    to='darkcave at shakespeare.lit'

    id='find45'>

<query xmlns='http://jabber.org/protocol/disco#info'

         node='files'>

<identity category='filesys' type='file' name='file1' />

</query>

</iq>

The fields in the identity stanza, are optional, but at least one
field MUST be provided. In this example, the responders will match its
files looking for the file names that contain 'file1' and are of the
size 1024..

Example XX. Returning with Matched Files


<iq type='result'

    from='darkcave at macbeth.shakespeare.lit'

    to='hag66 at shakespeare.lit/pda'

    id='find45'>

<query xmlns='http://jabber.org/protocol/disco#info'

         node='files/somefile'>

<identity category='filesys' type='file' name='file1'

hash='552da749930852c69ae5d2141d3766b1'/>

</query>

</iq>

A responding entity MUST include the name and the hash of the file.
Since more than one responder may respond with the same file, it is
strongly suggested that the initiator makes use of ranged file
transfers (as defined in XEP-0234), to speed up the file transfer.

Example XX. Finding files using Regular Expressions

Regular expressions may be use to find files that match the
expression. One or more fields can be used. The label attribute is
optional.


<iq type='get'

 from='hag66 at shakespeare.lit/pda'

    to='darkcave at shakespeare.lit'

    id='find46'>

<query xmlns='http://jabber.org/protocol/disco#info'

         node='files'>

<identity category='filesys' type='file' name='file1'/>

<x xmlns='jabber:x:data' type='get'>

<field var='ssn' type='text-single' label='Social Security Number'>

<regex>([0-9]{3})-([0-9]{2})-([0-9]{4})</regex>

</field>

</query>

</iq>

The XML character data of this element is the pattern to apply. The
syntax of this content MUST be that defined for POSIX extended regular
expressions, including support for Unicode.

The element MUST contain character data only (i.e., not
contain any child elements) and MUST NOT possess any attributes.

Any feedback would be greatly appreciated, I just want to know if I am
on the right track here.

GSOC is over, now what?

2011-09-11T21:21:00.000-07:00

Let's review what I did this summer:

* Stream Management

* IBB fallback for jingleFT

* Bidirectional Socks5 for jingleFT

I'm satisfied with what I have accomplished this summer. It was a wonderful experience. I would like to thank Yann (my mentor), Kevin (for putting up with all my annoying questions), Matt and Tobias.

My journey doesn't end here. I will continue to work in gajim. My goal is to have a full working implementation of jingle in gajim. For the bidirectional socks5, I had to rewrite a lot of stuff to make it work. Currently OpenSSL and proxies aren't working, but they will be fixed soon.

I've been a little distant this month because I've been partying like it is 1999

Bidirectional jingle socks5 file transfer in Gajim

2011-07-31T19:23:00.000-07:00

I've been busy these couple of weeks trying to make bidirectional jingle socks5 file transfer in Gajim. Currently in Gajim, it is the receiver who connects to the sender.

There are several class of interest here: SocksQueue, Socks5, Socks5Sender, Socks5Listener, and Socks5Receiver

Socks5Listener is the class in charge of binding to a port and accepting connections. SocksQueue manages the file transfer, it keeps track of when a transfer is completed or canceled.

Socks5Sender and Socks5Receiver, are in charge of sending and receiving the file, both of them are derived from the class Socks5. So, they are basically the same object with some minor differences.

As it can be seen from this picture, both objects are fairly similar, except for the receiver having the connect, do_connect, is_connected methods and the sender having the send_file method.

Those objects are also in charged of the socks5 negotiation and they keep track of it by using this state machine:

Status	Operation
0	About to connect
1	send version and auth types / initial read
2	read auth response / reply with desired auth type
3	send 'connect' request / read connect request
4	get approve of our request / approve connect
5	????? / read file
6	retrieve file / ????

There are more states, but these are the most important. What I have been doing so far is that have been modifying the objects so that they can emulate each other. Each object, for every state, will only do the thing that's after the "/" or before it. As I wrote the table, the operation in the left side of the "/" is what the receiver does and in the the right side of it, is what the sender does.

So far in my code, the receiver and the sender, do the operations in from state 0-2 . But then I realized that my approach was wrong all along. Instead of making the objects do both operations, Socks5 should be able to do both operations, and then the receiver and the sender can just inherit that behavior.

IBB fallback working

2011-07-10T20:20:00.000-07:00

The jingleFT branch of Gajim is now able to fall back to IBB when socks5 can't connect. Now, onto the testing phase. I have to write a test suite and maybe test IBB fallback with another client.

There isn't much to update. It works!

IBB status

2011-06-30T16:47:00.000-07:00

I've been meaning to write my report earlier, but I thought I would be finish with IBB by monday. At that point I had successfully transfered files over IBB doing a transport-replace.

The thing was that I was forcing Gajim to do a transport-replace right after the jingle session initiates. That was good for testing, but in the real world scenario, transport-replace would only come after Gajim recieves the following stanza:

<iq from='target@example.org/bar'
    id='hu3vax16'
    to='requester@example.com/foo'
    type='error'>
  <error type='cancel'>
    <item-not-found 
        xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/>
  </error>
</iq>

I encountered an array of minor problems related to the JingleFT branch, that have prevented me from successfully test for the IBB fallback. Hopefully they will be sorted out by the end of this week.

wiki ready

2011-06-20T15:43:00.000-07:00

The wiki for Stream Management can be found here.

Resumption is working

2011-06-15T19:08:00.000-07:00

That's a video with stream management resumption working. To tell what is going on, you need to put it in full screen.

All left to do is to test and then write some documentation on it.

Here's the diagram for the smacks class.

I will give a more detailed information about resumption in the wiki, and hopefully by next week I'll be implementing the jingle FT.

Starting to implement

2011-06-04T20:27:00.000-07:00

So, Gajim can basically send and request acks to a server. I haven't through fully tested it, but I will leave that for latter. Right now I'm focusing on resumption, which is harder because it changes the way Gajim connects to the server if it knows that resumption is accepted by the server.

For efficient acking the specification shows this example:

C: <enable/>
S: <enabled stanzas='5'/>
C: <message/>
C: <message/>
C: <message/>
C: <message/>
C: <message/>
C: <r/>
S: <a h='4'/>
C: <message/>
C: <message/>
C: <message/>
C: <message/>
C: <message/>
C: <r/>
S: <a h='9'/>

Which I find very weird that the server is the one who gets to set the interval in which the requests are sent. Since both, the sever and the client, can send and receive acks at any time, it doesn't makes sense that the server cares or has any say on how often the client sends acks.

So I made the judgment call to ignore this for now and let Gajim decide when it wants to send acks.

Into the first week

2011-05-28T13:42:00.000-07:00

So, the first week have been easy (as expected). I got everything done by Wednesday because I had finals. The rest of the week I focused on figuring out how to do the work for my second week.

What I did basically, is adding the ability to create all kinds of Acks to xmmpy.

Here's how xmmpy creates regular stanzas:

The Node class is in charge of the low level operations needed to make an stanza. The Protocol class has common operations used by the message, presence and IQ stanzas. Acks are different, so they don't need those kind of operations.

So my Acks class inherits from Node.

I tested it and it builds every Ack that are going to be needed by the client.

These are:

<a xmlns='urn:xmpp:sm:3' h=''/>
<r xmlns='urn:xmpp:sm:3' />
<enable xmlns='urn:xmpp:sm:3' />
<enable xmlns='urn:xmpp:sm:3' resume='true' />
<resume xmlns='urn:xmpp:sm:3' h='' previd='' />

So far so good.

Google Summer Of Code Starts Now!

2011-05-19T20:26:00.000-07:00

Hi! My name is Jefry Lagrange. I was selected to participate in GSOC 2011. I will be implementing Stream Management in Gajim, and some other goodies.

Here's my planned scheadule:

Week 1
--------------------------

Modify parser to create acks

Week 2
--------------------------

Create a class that will send acks, handle incoming acks and keep
track of the acks stanzas that have been handled.

Week 3
-------------------------

Analyze the source code, to know which parts should be modified in
order to implement stream management.

Week 4-5
--------------------------

Begin Stream Management implementation, negotiation, error handling,
stream closure.

Week 6
--------------------------------

Resumption.

Week 7
--------------------------------

Testing

Week 8
--------------------------------

Analyze the source code to know where and how file transfer is done.

Week 9
-------------------------------

Implement IBB fallback.

Week 10
------------------------------

Analyze the source code to know where and how the plugin system works.

Week 11-12
------------------------------

Extend the events in the plugin system to include things like outgoing
stanzas, etc..

I will update this blog every week's end with a report of my progress.