Notas de un Peregrino Articulos varios sobre la web, mozilla y firebug

New recording features for the MT-Simulator

The end of the Google Summer of Code is approaching and the Simulator have reached the Feature Completion stage. From now on, only minor bugfixes will be made untill the end of the project (August 16) when I have to submit the final code. From that on coding won't stop, is just that the GSoC ends. Probably the way things are done will be different and more feedback will be seek, but this project will surely continue.
And now the news about this release...

Notice
If you don't want to read the whole stuff and go right to the testing and trying fun stuff, just jump to the "Links! Links! Links!" section on the bottom

The new Recording features

Something that was very very needed was a more natural way to record the input files. Doing so programatically tends to be more dificoult and troublesome than moving your mouse around, so something had to be done. After some thinking and discussion with Felipe, I came up with a more comfortable way to record the files. I'll describe it briefly here, but you can alsgo skip it and see a demo after the break .

Basically the UI consists of a representation of the fingers on the screen. Moving them with the mouse represents the movement of the fingers on the multitouch surface.

First you click the Start Recording Button and choose where to save the final file. Once the file is specified, the number of fingers that will be on-screen has to be chosen. Pushing the Finger Button will add a finger to the screen, but beware because there's no way to remove a finger for now.

Doing the recording is pretty straight forward. All fingers are recorded in cycles, one at a time starting from the first one added and then going sequentially with the rest. The cycle starts again when the last finger is done.
Before starting to record the actual moves, you have to set the starting position of each finger. Doing so is pretty easy, drag the black-bordered (active) finger and then click on the Accept Button, then the next finger will be activated and so on. Once all the fingers are set, the recordings can start.

You will notice when the fingers are being recorded because a trace of the finger movement will appear on screen. If a mistake happens, the finger movement can be undone, and this can go as far as the first finger of the cycle. The recording cycles can be repeated until you decide to hit the save button.

When the cycle is finished (last finger moved) it can be played back, triggering the multitouch events on the page, with the Play button. This is in case some actions are needed in order to continue recording. Keep in mind that this is a replay of just the previous recorded cycle: you cannot do it twice and cycles don't accumulate.

When you decide the recording is done and click on the save button, the file will be closed and saved. Also, every element added for the recording will be removed from the screen.

Multitouch Landed on trunk!

For those who are wondering what does this means: Multitouch support is now on Minefield and will be included on the Firefox 4 Beta 3 that is going to be released soon. This also means that the custom build I had to upload on my previous post is no longer needed to try the extension

Links! Links! Links!

You can download the multitouch simulator 0.4 from AMO here. Remember to download the xpi and drag n' drop it on firefox, that will do for installation. In Firefox 3.6 you'll have to restart to see the efects, but in Firefox 4 betas the extension will load without restart If you find the bottom bar annoying, it can be hidden pressing Ctrl+Shift+U (or in OSX Cmnd+Shif+U).

If you just want to see this in action on a live demo, you can see the JavaScript library version here with the Firefox 4 beta 3 or greater. You don't need to install any addon, is a stand-alone version of the code running on a demo page

Also, here you have a list of available demos to try the multitouch support coded by Felipe Gomes:

• Image Crop
  Draw a rectangle to potentially "crop" an image with a pinch gesture
• Image Resize
  Shrink and Enlarge images with pinch gestures. With two simultaneous pinches you can resize two different images
• Touch Canvas
  Simple canvas that tracks your fingers
• Tracking Divs
  Two divs keep tracking of the fingers you have on the screen
• Fluids
  Fluid simulation following your fingers!

First stop on the road to completion

Some time has passed from the first post I made, and the extension now is more "usable". We showcased both the Firefox Multitouch and the Simulator project in the Mozilla 2010 Summit's Science Fair with my mentor, Felipe Gomes, and the people who passed by our booth were very interested in both projects! The input and the comments from the assistants were great!

Protocol optimized

After some tinkering with the old protocol, Felipe and I thought it was over sized and a little repetitive to work with, so we put it on a diet. Now it's more compact and kind of easier to work with, I'll show you/

This would be a sample file written with the new version:

{
  "frames":[
     {
       "positions":[
            {"id":"1","x":300,"y":300}
        ],
       "events":[
           {"id":"1","type":"MozTouchDown"}
       ]
     },
     {
       "active_duration": 5
     },
     {
       "positions":[
           {"id":"1","x":400,"y":400}
       ],
       "events":[
          {"id":"1","type":"MozTouchRelease"}
       ]
     }
  ],
  "fps": 10
}

The new rules are even simpler:

• frames
  A collection of Frame-Objects
• Frame-Object
  An object that has two properties to define the status of each frame:
  • positions
    A collection of objects containing the finger ID that the position belongs and x and y position: {ID, X_POS, Y_POS}
  • events
    Collection of objects containing finger ID and Event Type. Event Types are to be designed, but inital ones are the same as the implementation:
    • MozTouchDown
    • MozTouchUp
    • MozTouchRelease
    • MozTouchMove
• Frame-Object Types
  There are three Frame-Object types now:
  • Snapshot
    This is the type in where you define positions and events
  • Spacer Active
    A Frame-Object is of this kind when it's only property is "active-duration". This indicates that between two snapshots objects an animation should be done, and the duration is the value of "activ-duration"
  • Spacer Inactive
    This has the same structure as the active one, but instead the property is "idle-duration". This indicates that no transition should be calculated between two snapshots, it's a waiting time.
• fps
  The framerate that will be used to play the file

Changes:

• Frame-objects no longer have a fingers collection. This is now deduced from the positions IDs
• Events now are the same as the ones implemented in firefox. This is to simplify naming, and probably they'll soon change to something more "human" and less "techy".
• The Frame-Objects have 3 subtypes now: Snapshot, Spacer Active, Spacer Idle.

Demo Available

As I promised last time, the code is ready for some demoing!

As the multitouch part is not yet available on firefox latest builds you'll have to use a special build of firefox to try this stuff. Here you'll find a compiled version for your system:

• Mac OSX (tested on 10.6, should also work on 10.5)
• Windows
• Linux (not here yet, still have to compile this)

Now that you have the correct build to test the extension is time to install it. I would strongly recommend you to create a new profile, it's a good practice when you're trying new stuff that may explode .

You can grab here the extension compiled as an XPI, and here some good sutff: you won't need to restart firefox to try it out! As a side-effect of coding the extension with Jetpack, and having to use the latest firefox build for the multitouch part, we end up having the best setup ever: restartless addons enabled! To install the addon, just download it and drag-n-drop it on firefox.

But we need some demos to try the addon, so here you have a list of available demos to try multitouch coded by Felipe Gomes:

• Image Crop
  Draw a rectangle to potentially "crop" an image with a pinch gesture
• Image Resize
  Shrink and Enlarge images with pinch gestures. With two simultaneous pinches you can resize two different images
• Marcio Drawing
  A demo that lets you draw with your fingers. Touching a color in the pallete, changes the drawing color.
• Pong (not much usefull in for this addon)
  A simple implementation of the classic Pong game.
• Touch Canvas
  Simple canvas that tracks your fingers
• Tracking Divs
  Two divs keep tracking of the fingers you have on the screen
• Fluids
  Fluid simulation following your fingers!

And here are some sample input files:

• Sample 1: A simple pinch gesture.
• Sample 2: Two simultaneous simple pinch gestures.
• Sample 3: Cross-drawing (this is designed for the Marcio Drawing Demo)

Using the simulator is pretty simple, it just has three buttons:

The first one lets you load an input file (as the samples I'm providing) to be processed. The second one opens a kind of "console" so you can write a input file on the fly and then run it with the third button, as you can see here:

Where to go now

The next steps I think will be the harder ones. What is pending to be added is a way to simulate multitouch input with just one pointer. That means, how to make the extension itself write the input files instead of doing it by hand. I still have to come up with a UI for that, and any suggestion/idea is welcome. If you think you can collaborate on that stuff, don't hesitate to contact me on twitter or leaving a comment.

Feedback

I would really appreciate if some (or all!) of you try this extension and reports issues/enhancements (you can do it here), that would be priceless. Of course any thing you want to ask you can reach me in #jetpack on irc.mozilla.org,  I go by the name of "peregrino", or by leaving a comment here.