Tuesday, December 9, 2008

Remoting with IProgressMonitor

Recently we added optional progress monitoring for remote requests to the Net4j Signalling Platform and I'd like to take this opportunity to explain some of the exciting aspects of Net4j. You can use Net4j to implement synchronous remoting applications as well as asynchronous messaging applications. Depending on the number of Net4j layers you are using you can work buffer-oriented or stream-oriented.

Another multi-purpose technology

The heart of Net4j is a fast and scalable, Java NIO-based buffer switching technology which is completely non-blocking and allows for multiple levels of buffer multiplexing. On top of this buffer-oriented transport layer there is an optional stream-oriented protocol layer which is used to implement application protocols. The framework automatically ensures that multiple such protocols can be multiplexed through a single physical transport connection like a TCP socket and that the transport type can be transparently changed to other mediums like HTTP or JVM.

I think the idea behind Net4j becomes clearer when we look at an example. Let's develop a small protocol to upload files to a remote server with full support of IProgressMonitor over the network. In Net4j this is a single signal (aka communications use-case) of a protocol and we need to provide implementations for both sides of the protocol. We start with the client-side request implementation.

Now the application can create instances of this request and call the send() method on them. The framework in turn calls the requesting() and confirming() methods when the transport layer is able to handle the data. The following example method encapsulates the sending process. Notice how the IProgressMonitor is converted to an OMMonitor.

This simple method can now be used in a workbench action. The transferFile() call is wrapped in a Job to make it run in a background thread and to integrate with the Eclipse Progress view.

The complete source code of this action class is available via CVS. Now we only need a server that is able to receive and handle our request. We start with the server-side request implementation (called an indication).

When the data of an UploadRequest arrives atthe server the Net4j framework creates an instance of our UploadIndication and calls the indicating() and confirming() methods appropriately. Therefor we need to register a signal factory, i.e. a customized protocol. In our case we create the protocol instance on-the-fly inside of the protocol factory.

If the server was expected to run in OSGi/Equinox the only missing thing would be the contribution of this protocol factory to an extension point.

But we want to look at a simple stand-alone version of our upload server.

The complete source code of the server class is available via CVS. To enhance the maintainability of our upload protocol some protocol constants are defined in a common interface. This makes it particularly easier to add new signals.

Now we just need to start the server and watch the console.

And finally we start a runtime Eclipse application with our UploadClientAction deployed. When you click this action a file selection dialog is displayed. The chosen file will be uploaded to the server and stored there in a temp folder. The progress of the upload operation is properly reflected in the Eclipse Progress view and can even be cancelled at any point in time.

This is only one example of the many things you can do with Net4j. Please note that the actual client/server communication in Net4j has some characteristics that differ from other remoting technologies, among them:
  1. There are no discrete marshalling/un-marshalling phases. As the client writes data to a stream this data is internally filled into fixed-size buffers and passed to the transport layer as soon as they are full enough. As a consequence the server already starts reading and processing that data while the client is still busy sending more data.
  2. Multiple virtual channels can be multiplexed through a single physical transport connection. Each such channel can be associated with its own signal protocol and then be used to multiplex arbitrary numbers of signals into both directions.
  3. The client and server roles do only apply while establishing a physical transport connection. Once connected both sides can open virtual channels at arbitrary times and send or receive data, possibly through signals.
A larger example of a Net4j application protocol is the one that connects EMF models with a CDO model repository. It is also an asymmetric protocol and consists of a client-side implementation and a server-side implementation.

I hope some of you found this interesting. Maybe I have some more ideas for articles about Net4j in the future...


  1. Hi Eike
    just as a follow up I went and had a look at the net4j page and tried to follow some of the links such as documentation - net4j user interface or net4j architecture and get

    Permission error
    From Eclipsepedia
    Jump to: navigation, search

    You do not have permission to do that, for the following reason:
    The action you have requested is limited to users in the group user.

    ... might confusing and not so helpful to the interested

  2. Have you talked with Scott Lewis about how this might work in conjunction with ECF? I think that'd be interesting.

  3. Varioustoxins, The root cause is that these links are not yet filled with article content, which automatically activates the wiki "edit" mode. For this mode you must be logged in. For now I will place some empty stubs to avoid the error message and then come back to fill more meaningful descriptions later.

    AlBlue, I not only spoke with Scott, rather together we already implemented an ECF provider on top of Net4j. You might want to read this bugzilla. It might also be a good idea to further nag Scott and me so that we finally publish the results. This way or that way we're both interested in your feedback!

  4. Hello Eike,

    I was looking through the web trying to find a Remote ProgressMonitor and I stumble upon this blog...

    This looks quite interesting. I was wondering if OMMonitors can handle this use case?

    I have a client and a server using ECF-OSGI framework to remotely communicate to each other. The Server job does something to the request of the client and sends back the result. I want to have the server to communicate the progress to the client.

  5. Hi Kevin,

    This is exactly what Net4j's remote progress monitoring has been design for. I don't know whether ECF has a similar mechanism. It's best to aks on their mailing list. However, there's an old integration between ECF and Net4j, but it hasn't been maintained for years.