Introduction

OSGi is a lightweight Java framework to build component-based service-oriented applications. The MUSIC middleware is built on top this framework due to its advantages.
However OSGi lacks of support for ubiquitous computing and distributed environments. There is some support for service discovery in the OSGi specification (e.g. UPnP) and some non-standard OSGi bundles to set up a distributed OSGi platform (e.g. R-OSGi).
The MUSIC consortium decided to build our own communication services because the available approaches did not accomplished all of our requirements::

• It must provide 2 main features: remote binding and service discovery.
• The communication services must be fully integrated into the OSGi framework.
• These services must be open and reusable for MUSIC and any other OSGi-based project.
• The implementation must be lightweight. MUSIC targets mobile devices.
• It must be extendable based on plugins.
• It must be open source with LGPLv2.1.

Distributed OSGi

Each OSGi framework includes a service repository to support a Service-Oriented Architecture. An OSGi service is a Java object instance, implementing a Java interface (or several Java interfaces), registered in the OSGi framework with a set of properties. These OSGi services are shared among all the OSGi components running in the OSGi framework but they are constrained into the same Java runtime environment.
In order to overcome this limitation, MUSIC has built a set of communication services which make possible share OSGi services into several OSGi frameworks by:

• Discovering the OSGi services published by other OSGi frameworks
• Importing a remote OSGi service (e.g. previously discovered)
• Invoking a remote OSGi service in a transparent way (as if it were a local OSGi service)

The main contribution of the MUSIC communication services is the ability to set up a distributed OSGi framework but making completely transparent the distribution.

The MUSIC communication services

There are 3 communication services (which are also OSGi services) defined by the MUSIC architecture:

1. The remote manager which is able to export/unexport and import/unimport services.
2. The discovery manager which publish/unpublish services and provides mechanisms for searching services.
3. The service classifier which provides additional properties for a service according to the service classification.

The org.istmusic.mw.communication bundle provides the Java interfaces for these services and the implementation for the remote and discovery managers.
The communication services are not bound to a specific remote binding technology (e.g. plain sockets, web services, ...) or service discovery technology (e.g. SLP, UPnP, ...). The communication plugins implement a specific technology support and the middleware user might select which technologies to be used by adding this plugins to the middleware (which are actually OSGi bundles too).

The remote manager

The remote manager is an OSGi service implementing the following Java interface:

package org.istmusic.mw.communication.remoting;

import org.istmusic.mw.communication.CommunicationException;
import org.istmusic.mw.communication.ServiceDescription;
import org.istmusic.mw.communication.ServiceInstance;

/**
 * Interface of the Remote Service Manager. The Remote Service Manager will implement a
 * pluggable mechanism to accept different remoting technologies. Each plugin implements
 * the IRemotePlugin interface.
 * 
 * @author Telefonica I+D
 * @author Yun Ding (yun.ding@eml-d.villa-bosch.de)
 */
public interface IRemote {
	/**
	 * Export a local service to make it accessible from other MUSIC node
	 * 
	 * @param serviceInstance Information of the local service instance
	 * @return Service description of the exported local service
	 * @throws CommunicationException
	 */
	public ServiceDescription exportService(ServiceInstance serviceInstance) throws CommunicationException;
	
	/**
	 * Unexport a local service (previously exported)
	 * 
	 * @param description Service description returned by exportService method
	 * @throws CommunicationException
	 */
	public void unexportService(ServiceDescription description) throws CommunicationException;
	
	/**
	 * Import a remote service so that it's accessible locally in a transparent way (as it were local)
	 * 
	 * @param description Service description of the remote service
	 * @return Local reference to the imported service (through the service proxy)
	 * @throws CommunicationException
	 */
	public Object importService(ServiceDescription description) throws CommunicationException;
	
	/**
	 * Unimport a remote service
	 * 
	 * @param serviceId Service identification (field of the service description)
	 * @throws CommunicationException
	 */
	public void unimportService(String serviceId) throws CommunicationException;
	
	/**
	 * Return a remote plugin for a specific protocol. If the protocol is null, it will
	 * return the default remote plugin.
	 * 
	 * @param protocol
	 * @return It could be null in case that there's no remote plugin for the specified remoting protocol
	 */
	public IRemotePlugin getRemotePlugin(String protocol);
}

The remote manager provides mechanisms to:

• Export a local service. This local service might be a plain Java object or a local OSGi service. Once the service is exported, other OSGi frameworks will be able to import it.
• Unexport a local service (previously exported).
• Import a remote service. The remote manager will register a local OSGi service based on the service description characterising the remote service. The service description will contain enough information to infer the java interfaces implemented by the remote service. This approach let us invoke a remote service as if it were local.
• Unimport a remote service (previously imported).
  

Although it is possible to use the remote manager as an OSGi service, typically it is not required because it is configured to:

• Export any local service with the following OSGi service property: remote-enabled=1
• The discovery manager is linked to the remote manager to import any discovered service.
  

The remote plugins provide support for different remote binding technologies. The remote plugins implement the following interface:

package org.istmusic.mw.communication.remoting;

import org.istmusic.mw.communication.CommunicationException;
import org.istmusic.mw.communication.ServiceDescription;
import org.istmusic.mw.communication.ServiceInstance;

/**
 * Interface for a "Remote Service Manager" plugin which implements a specific
 * remoting technology 
 * 
 * @author Yun Ding (yun.ding@eml-d.villa-bosch.de)
 * @author Telefonica I+D
 */
public interface IRemotePlugin {
	
	/**
	 * Remoting protocol implemented by the plugin 
	 * 
	 * @return
	 */
	public String getProtocol();

	/**
	 * Create a skeleton (server-side proxy)
	 * 
	 * @param serviceInstance
	 * @return Service description
	 * @throws CommunicationException
	 */
	public ServiceDescription createSkeleton(ServiceInstance serviceInstance) throws CommunicationException;
	
	/**
	 * Destroy the skeleton associated to an exported service
	 * 
	 * @param serviceName
	 * @throws CommunicationException
	 */
	public void destroySkeleton(String serviceName) throws CommunicationException;
	
	/**
	 * Create a stub (client-side proxy)
	 * 
	 * @param serviceDescription
	 * @return
	 * @throws CommunicationException
	 */
	public Object createStub(ServiceDescription serviceDescription) throws CommunicationException;
}

The remote plugins basically creates the skeleton in the server-side and the stub in the consumer-side. These remote plugins typically make extend use of Java reflection and specifically of the java.lang.reflect.Proxy class. A dynamic proxy class is a class that implements a list of Java interfaces (the service interface) and offers a handler to process any invocation to that class (in order to carry out the remote communication between Java objects).

The discovery manager

The discovery manager is an OSGi service implementing the following Java interface:

package org.istmusic.mw.communication.discovery;

import org.istmusic.mw.communication.CommunicationException;
import org.istmusic.mw.communication.ServiceDescription;

/**
 * The interface IDiscovery is provided by the Discovery Service for the 
 * publication and discovery of devices and services. 
 * 
 * @author Yun Ding (yun.ding@eml-d.villa-bosch.de)
 * @author Telefonica I+D
 */
public interface IDiscovery {
	/**
	 * Publishes a service with the specified service description
	 * 
	 * @param description Description of the service to be published
	 * @return True if the service was published successfully
	 */
	public boolean publish(ServiceDescription description);

	/**
	 * Unpublishes a service with the specified service description
	 * 
	 * @param description Description of the service to be published
	 * @return True if the service was unpublished successfully
	 */
	public boolean unpublish(ServiceDescription description);

	/**
	 * Searches for services
	 * 
	 * @param attr Search attributes used to filter the services
	 * @return Array of ServiceDescription for each discovered service
	 * @throws In case of an error during the search
	 */
	public ServiceDescription[] search(SearchAttributes attr, String protocol) throws CommunicationException;
}

The discovery manager provides mechanisms to:

• Publish a local service so that it can be discovered by other discovery manager instances.
• Unpublish a local service.
• Search remote services. The search can be filtered according to some attributes or it could search for any published service.

The discovery manager is configured to discover all the remote services by default. It initiates a thread which search for remote services in time intervals. This process is rather convenient to make it transparent in the general case.
The discovery manager is fully integrated with the remote manager in order to simplify both the publication and search of services:

• When a service is exported, automatically is published
• When a service is unexported, it is unpublished as well.
• When the discovery manager discovers a service, it automatically imports that remote service. This step will only work when the consumer-side has the java interfaces provided by the remote service.

The discovery plugins provide support for different service discovery technologies. The discovery plugins implement the following interface:

package org.istmusic.mw.communication.discovery;

import java.util.Iterator;

import org.istmusic.mw.communication.CommunicationException;
import org.istmusic.mw.communication.ServiceDescription;

/**
 * Interface for a "Service Discovery Manager" plugin which implements a specific discovery technology
 * 
 * @author Yun Ding (yun.ding@eml-d.villa-bosch.de)
 * @author Telefonica I+D
 */
public interface IDiscoveryPlugin {
  /**
   * Publishes a service with the specified service description.
   * 
   * @param description Service description to be published. It will be transformed to the protocol-specific service description
   * @return 
   */
	public boolean publish(ServiceDescription description);
	
	/**
	 * Unpublishes a service with the specified service description.
	 *  
	 * @param description Service description to be unpublished. It will be transformed to the protocol-specific service description
	 * @return 
	 */
	public boolean unpublish(ServiceDescription description);
	
	/**
	 * Searches for services using a particular service discovery protocol
	 *  
	 * @param attr Search attributes used to filter the services
	 * @return An iterator over the found service descriptions. 
	 * @throws CommunicationException
	 */
	public Iterator search(SearchAttributes attr) throws CommunicationException;
	
	/**
	 * Returns the discovery protocol supported by this discovery plugin
	 * 
	 * @return The discovery protocol name
	 */
	public String getProtocol(); 
}

This interface is very similar to the IDiscovery interface.

Example

The following example describes how to use the communication services in the simplest situation. There are 2 different nodes:

• A provider-side node. This node has a local OSGi service which is exported and published so that other nodes can consume it.
• A consumer-side node which discovers the service published by the provider-side node and an OSGi application will use and invoke the remote service.

This example will use the default communication plugins:

• The remote socket plugin which is a simple but high-performance implementation for binding the consumer and provider sides by the serialization of Java objects and its delivery by the network to a specific port and IP address.
• The discovery SLP plugin which implements the Service Location Protocol standard based on multicast messages for reporting published services as well as for querying for services.

The following figure represents the OSGi bundle configuration in each node. The green blocks correspond to the application bundles.


Both nodes share the "Common interface" bundle. This bundle will provide the Java interface/s provided by the service and it is required in both nodes. The Java interface is:

package org.istmusic.example;
public interface IHello {
	public void sayHello();
}

The "Service provider" bundle implements the IHello interface and registers it as a local OSGi service. In order to export and publish the service automatically, it is included the service property "remote-enabled=1" in the OSGi registration. This process can be achieve rather easily with OSGi declarative services:

<?xml version="1.0"?>
<component name="hello.service" immediate="true">
  <implementation class="org.istmusic.example.impl.HelloService"/>
  <property name="remote-enabled" value="1"/>
  <service>
    <provide interface="org.istmusic.example.IHello"/>
  </service>
</component>

In the consumer-side node, the "Consumer application" will have a dependency with the IHello service. The communication services make transparent the distribution stuff and it only requires to retrieve the service from the local repository. Declarative services are also very convenient to activate the application/component only when the service is available:

<?xml version="1.0"?>
<component name="hello.application" immediate="true">
<implementation class="org.istmusic.example.application.HelloApp"/>
<reference name="hello.service"
  interface="org.istmusic.example.IHello"
  cardinality="1..1"
  bind="setHello"
  policy="dynamic"
/>
</component>

The execution steps are the following ones:

1. Consumer-side: The "consumer application" bundle registers the application component but it is not activated until the hello service dependency is satisfied.
2. Provider-side: The "service provider" bundle registers the service (with the property remote-enabled=1)
3. Provider-side: The remote manager detects the registration of an OSGi service with the remote-enabled property. It triggers the export and publication of the service according to the default communication plugins:
1. The socket plugin will create the skeleton to attend any request to this service. An IP port will listen to these requests.
2. The SLP plugin will publish the service description according to the SLP standard. The service description will be propagated via multicast messages and include information about: the Java interface which implements the service, the port and IP address linked to the skeleton, and other service properties.
4. Consumer-side: The discovery manager automatically detects a new service description (by using the SLP plugin) and asks the remote manager for importing the service.
5. Consumer-side: The remote manager imports the service by:
1. Asking the socket plugin to create a stub. This step is only possible if the Java interface/s specified by the service description are available in the consumer-side. In this example, the "Common interface" bundle provide such interface.
2. Registering this stub as an OSGi service (implementing the IHello interface)
6. Consumer-side: The declarative services runtime will detect the new IHello service, which satisfies the application dependencies, and activates the component/application.

The communication services and the MUSIC applications

The MUSIC applications are modelled with component types and plans to support the variability, which is rather convenient in dynamic environments like those ones targeted by the MUSIC project. The MUSIC applications profit from the communication services regarding the following features:

1. Incorporate services. Services can be discovered, imported and incorporated as possible realizations of some component types. Afterwards, the adaptation middleware would be able to select those realizations so that the application consumes services.
2. Hosting of services. A MUSIC application can provide services which could be locally-constrained or could be widely available by exportation and publication.
3. Distributed applications. Applications are typically modelled as a composition of component types where the realizations of each component type might be in the same node or distributed among several MUSIC nodes.

Incorporate services

A MUSIC application is modeled as a composition of component types where the realizations of each component type might be a component, a service or another composition.
A service dependency is modelled in the composition as a new component type, where that component type has a service type with a specific service description which provides information about:

• The service classification
• The Java interfaces provided by the service
• The default properties (in case that it is not possible to retrieve these properties from the service description).

The Service Planner is a MUSIC OSGi service which implements the IServiceClassifier interface (of the MUSIC communication services). The service planner takes care of 2 areas:

• Adding new properties to the service description and based on the service classification. Whenever a service is discovered by the discovery manager, it will try to aggregate additional properties to the service description by invoking services implementing the IServiceClassifier. The service planner will find which service plans fit to the specified service classification. If any valid service plan is found, it will provide:
• Which Java interfaces are provided by the service
• Add default value properties if the service description does not include those property values.
• Add an special property: music.service.plan=true to indicate that the current service description fits to a service plan.
• Creating service plan variants. The Service Planner listens to the registration of any OSGi service with the property: music.service.plan=true. Whenever it is found, the service planner will create a service plan variant according to the OSGi service properties.
  

The execution process to incorporate a service is:

1. The discovery manager detects a new service.
2. The discovery manager asks the service planner for adding new properties. If the service matches a registered service plan, it will add new properties to the service description. If not, the service description will be maintained.
3. The discovery manager requests to the remote manager to import the discovered service.
4. The remote manager imports the service by creating a stub and registering this stub as a local OSGi service (the OSGi service properties will be the service description properties).
5. The service planner will detect a new OSGi service with the property music.service.plan=true. It will create a service plan variant according to the OSGi service properties.
6. The adaptation middleware triggers an adaptation due to the new service plan variant. The application might be reconfigure to consume the service.

Hosting of services

An application developer will specify a hosting of service by:

• Adding a service type (formerly known as port type) to the application type
• Including a service description to this service type
• Specifying a provided Java interface/s in the service type
• Listing a set of property names which will be included in the service description

The hosting of services can be just modelled at the highest level (application level); however, the component providing that service can be a valid realization at any level of the composition hierarchy. This is achieved by means of port delegation.
The property values to complete the service description are taken from the realization. These properties can be used for: a) support for the communication services (e.g. specify that a service must be exported and published), b) QoS values, and c) other purposes.
The following figure represents a MUSIC application which hosts a service. The application/component types are the blue blocks while the realizations are the grey ones.



The previous model defines two property names: remote-enabled (to export and publish the service) and precision (a QoS property). These properties will be included in the service description. The model supports 3 different service descriptions depending on the variant selected by the adaptation middleware. The property values are evaluated in runtime (during the application reconfiguration) from the property predictors associated to each realization. The properties in the higher levels take precedence over the same properties in lower levels (e.g. the remote-enabled property defined in the atomic realization 3 is not consider because the same property was defined in the composition realization which corresponds to a higher level in the hierarchy.

	remote-enabled	precision
Atomic realization 1	true	low
Atomic realization 2	false	medium
Atomic realization 3	false	high

The process executed to host a service is:

1. The adaptation middleware triggers an adaptation and selects to reconfigure an application (which hosts a service).
2. The InitBatch in the configuration planner will determine if the application type has service types valid to host services. If that is the case, it evaluates the property predictors to complete the full list of properties required for the service description. There are 3 different possibilities:
1. The application has just been launched (the application did not host the service). The configuration planner creates a ServiceHostStep with all the information related to the service to be hosted.
2. The application was already launched and some service property has changed (probably the realization changed). The configuration planner instantiates a ServiceUnhostStep to remove the hosted service (which is not valid anymore) and instantiates a ServiceHostStep to host the service with the new configuration.
3. The application was already launched and no service property has changed. Then, no step related to hosting of services is required.
3. The configuration executor receives a ServiceHostStep. It will retrieve the Java object associated to the component which provides the service and invoke the kernel OSGi factory to register an OSGi service with all the evaluated service properties. And additional property is added by the configuration executor: music.service.plan=true, so that this hosted service might be incorporated into an application.
4. The remote manager (in the communication services) will discover this new OSGi service if the property "remote-enable=true". In that case, it will export and publish the service.
5. The service planner will discover the OSGi service, with the service property: music.service.plan=true. It will look up any service plan compatible with this service (according to the service classification property). If any valid service plan is found, it will create a service plan variant.

Distributed applications

The configuration planner will detect which components require a remote binding during the ConnectionBatch. When a remote binding is required, the configuration planner will create the appropriate steps to export the provided interfaces in the remote component and import them in the local component, and vice versa. The configuration executor will make use of the remote manager directly to export/import the components.

Linux

MUSIC communication services can make use of any service discovery technology, with the appropiate discovery plugin. The middleware currently supports SLP and UPnP, being SLP the main discovery protocol. Linux users have to take into account the following:

• If your kernel supports IPv6, you have to add the property "java.net.preferIPv4Stack=true" to the configuration, because jSLP (the SLP library used by MUSIC) does not support IPv6.

• If you do not have root permissions, SLP will not be able to bind to port 427, as port numbers between 0 and 1023 are reserved. You can change this port by setting the property "net.slp.port" with a port number greater than 1024, but this will make it incompatible to standard SLP nodes. For that reason, all the MUSIC nodes you want to interact with must have the same port number. As an alternative, you can use programs that allow opening reserved ports to non-root users, like "privbind".

• Some JVMs return 127.0.0.1 as the local IP address. If this is your case, you have to edit the /etc/hosts file to point your hostname (as defined in /etc/hostname) to your real IP address. If you do not have the privileges to edit these files, set the property "net.slp.interfaces" with the real IP address (this might lead to some bugs).

Conclusion

The MUSIC communication services are a set of OSGi services which are valid to enhance the local vision for services of the OSGi standard. It is composed by a set of OSGi bundles which do not have any dependency with other parts of the MUSIC middleware (except the Service Planner). As a result, these bundles can be used in any OSGi project.
The communication services guarantee a solid integration with the OSGi framework, extensibility based on plugins, and transparency to avoid the distribution complexity.