The Short Story
Finally the Gemini Management project has a Git repository. The SVN repository is used no more. You can use the following repository URLs to clone the project's code:
- git://git.eclipse.org/gitroot/gemini.management/org.eclipse.gemini.managment.git
- ssh://git.eclipse.org/gitroot/gemini.management/org.eclipse.gemini.managment.git
- http://git.eclipse.org/gitroot/gemini.management/org.eclipse.gemini.managment.git
You can access the repository on the web here.
For those not interested in details, that's it. The more curious can keep on reading.
And Now The Long Story
From its inception, the Gemini Management project used to use a SVN repository. Exactly as all other Gemini projects do. Except Gemini Web, which switched to Git some time ago.
But things are changing. Now Git is the official revision control system for Eclipse and most projects migrate to it. I will not enter the endless polemics of which revision control system is better, or more powerful, or has better tooling and is more usable. The fact of real life is that in some point in time all Eclipse projects will change to use Git. And I myself prefer this point to be sooner than later. Because I find it rather annoying to use three different tools for three different revision control systems - I use CVS for Eclipse Orbit and Eclipse documentation, Git for Equinox projects and Virgo, and I used to use SVN for Gemini Management. That is why I decided to move Gemini Management to Git. Now I use one tool less :)
Migrating itself is very easy, actually. The Eclipse Webmaster creates the Git repository on the filesystem for you. Then you have to ssh to git.eclipse.org to initialize the new repository. Actually, in the general case you do not have rights to execute any command on this host, except initrepo. Finally you clone the empty repository on your local machine, copy the project sources in it, commit and push. This is all you have to do. But if you want to keep the history as well, it is a little bit more complicated.
I will tell you how I kept the history. I'm not a Git guru, probably there's a better and cleaner way to do it. But at least mine works fine. So, I initialized the repository the Webmaster had created on git.eclipse.org and cloned the empty repository locally. Then, I used git-svn to copy Gemini Management sources from the remote SVN repository to a new local git repository. Exactly how to do this you can find here. I created the authors file, created a new folder for the repo and executed
git svn init <svn_repo_url> --no-metadata
git config svn.authorsfile <path_to_authors_file>
git svn fetch
Now I had a new git repo containing everything from the SVN repo, including the history. I had to push this to the existing remote Git repo. So I copied the contents of the .git/config file from the empty Git project I have previously cloned to the real one. I also copied the .git/HEAD reference from the empty to the real. I removed the contents of .git/refs folder in the real project and copied the contents of .git/refs from the empty to the real. I also deleted .git/svn folder from the real project. Probably this copying was a kind of hack, but it worked. I git fetched, git pulled and then git pushed. And Voila! - the source code got to the remote git repo of Gemini Management. Then I git fetched and git pulled in the empty local project and I got the Gemini Management code along with the history in it. You can get it too from the URLs in the beginning of this post.
Anatomy of a Gogo command
In the built-in Equinox console commands are provided by classes extending the CommandProvider interface. All commands should be public methods, starting with underscore and accepting as an argument a CommandInterpreter object. A little bit restricting, isn't it? Well, now comes the new shell, with the Gogo-type commands, which have none of these restrictions.
Gogo commands are provided by an arbitrary class (no interfaces to implemet!). The commands should be public methods of the class and they may take arbitrary arguments. The command providers are registered as services with two specific properties:
osgi.command.scope - specifies the scope of the commandsosgi.command.function - lists the names of all commands in the provider (i.e., the names of the public methods in the class)
Following is a very simple example of a Gogo command, which says "Hello world!".
pulblic class HelloCommand {
public void sayHello() {
System.out.println("Hello world!");
}
}
The code to register this command is:
...
Dictionary<String, Object> properties = new Hashtable();
properties.put("osgi.command.scope", "myscope");
properties.put("osgi.command.function", new String[] {"sayHello"});
context.register(HelloCommand.class, new HelloCommand(), properties);
...
In this way you can register existing classes with public methods as command providers. Sometimes this could be very useful.
Converters and Formatters
As I mentioned, the commands may accept arguments of arbitrary types. Probably you wonder how the strings, read from the input stream, are converted to the command argument type? This is possible because of the special converters which the Gogo shell introduces.
The converter is a class which knows how to convert the value, read from the command line, to the actual argument. For example, your command may accept a Bundle object as an argument. Assume you want to call the command with the bundle id or the bundle symbolic name. You write a converter, which looks up the Bundle object in the BundleContext by the id, or in the PackageAdmin service by its symbolic name, and returns the Bundle object.
The converters are registered as services. A converter should implement the interface org.apache.felix.service.command.Converter. The interface has two methods - convert and format. When the Gogo shell executes a command, it calls all registered converters, passing the input arguments to their convert methods. It calls the command with whichever converted object matches its argument type.
The output of a Gogo command can be an object of arbitrary type. The shell itself cannot know how to display all possible objects. So Gogo uses again the converters, but their format method. For example, your command may return a Dictionary object. You may register a converter, which knows exactly how to format and output the values in the Dictionary. After Gogo executes the command and gets the result, it calls all registered converters on it.
You may register a converter implementation which implements only the convert method, only the format method, or both - it depends on your command. Gogo provides some default converters, so it is not always necessary to provide your own.
Customizing the Tab Completion
As explained in the previous post, the new shell provides tab completion for commands names, variables names and files names. It is possible that the command has specific input and in this case it would have a better knowledge how to complete it. For this case the shell supports custom completers. A custom completer should implement the interface org.eclipse.equinox.console.common.Completer. It has one method - getCandidates, which gets as an argument the whole command line and the current position of the cursor in it, and returns a map with all completion candidates. The keys are the candidates, and the values are the start positions for the completions. The completers are registered as services.
Using Commands
The command usage in the new shell is not much different from whay you have seen in the built-in Equinox console. However, there are minor differences, mainly due to the fact that the new shell is based on the Apache Gogo shell and its commands are a little bit different.
Although the differences are much more evident when developing commands, they still could be found when using them. First of all, with Gogo comes the notion of scope. Each command has its scope, within which there should be no other command with the same name. In practice, if there are registered several commands with identical name and scope, the last one registered wins.
However, scoping solves to some extent the problem with identical commands names. In the old console all commands should be with different names - otherwise Equinox does not guarantee which one it will execute. With the nothion of scopes each bundle could provide a unique scope and then has commands with the same names as commands in other bundles.
When calling commands the scope is represented by a prefix of the command name, separated from it with a column. You may call a command without prefixing with its scope, but do this only when you are sure that there is no command with the same name in another scope. Or else the other may get executed. The Gogo scope takes priority over all other scopes, so if a command is called without a scope, it is first searched within the Gogo scope, and then in the others.
Equinox comes with a number of commands and there are also other Eclipse bundles, providing console commands. There are also many custom bundles, which provide commands. All these are not Gogo commands and Gogo does not recognize them. That is why the new shell provides an adapter for these commands, so that all commands that used to work in the built-in Equinox shell will continue to work in the new one. In the new shell the adapter registers them with scope equinox.
If you want to disconnect from a telnet or ssh session, use the disconnect command. The exit command will directly stop the Equinox framework, which could be an unpleasant surprise.
The Help Commands
Gogo comes with its own help command. When called with no arguments, it lists all available commands, together with there scopes. If a command name is passed as an argument to the help command, it displays help for the given command, if such is available. However, Gogo help command does not recognize the "adapted" legacy Equinox commands. That is why there is also another help command, with scope equinox, which does recognize them. It has also accepts two options:
-legacy - lists only the "adapted" commands-all - lists all commands, both new and "adapted"
If used without arguments it behaves lake the Gogo help command and lists the new commands.
If you want to get help for a particular "adapted" command, you should use equinox:help -legacy and the name of the command, without scope. This will work only for legacy commands, which provide help.
Instead of using equinox:help, you could use man. It has exactly the same behavior, and you could omit its scope (which is equinox), because there is no other command with this name currently.
Tab completion
A nice feature of the new shell is command completion. This feature is available only when connecting through telnet and ssh. Command completion is available for command names, session variables names and file names. The variables should be previously defined in the session. The completion functionality is activated by pressing the tab key. If there is only one possible candidate, the current string is completed with it. If there are more candidates, they are listed in a column and may be iterated by pressing tab multiple times until the desired candidate is reached.
Configuring the New Shell
Last time I introduced the new shell and described how you can start using it. Now I will explain in more details the esoteric formulas in the configuration file for running the shell.
Let me start with the config.ini file.
The first entry there, osgi.bundles, specify which bundles should be started along with the start of the Equinox. Obviously, in our case these are all bundles, related with the shell. This is in case you don't want to install and start them manually after running the framework.
Not all of the bundles I have included in the configuration are absolutely necessary. If you want to have the full functionality of the shell, you do need all of them. But if you need only the shell or the shell with telnet access, you do not need most of them. In this partial scenario you will have to include in the configuration the three gogo bundles, and the org.eclipse.equinox.console.supportability bundle. All the rest are necessary only for the ssh support.
As I mentioned in the previous post, Equinox framework comes with a very basic console built into it. In order for the new shell to function properly, this default console should be disabled. This is exactly what the property osgi.console.enable.builtin set to the value false does.
The property osgi.console specifies the telnet port, on which the console listens. If you do not want the console to open a telnet port, do not specify this property. The value of the property could be just the port number, or may have the format <host>:<port>. In the latter case the port is opened only on the specified network interface. This is used to increase security. If you open the telnet on <localhost>:<port> or <127.0.0.1>:<port>, then the shell will be accessed on only from the localhost - remotely there will be no access. If the host is not specified, then the port is open on all network interfaces and connection is possible from remote hosts.
The property osgi.console.ssh specifies the port, on which the console listens for ssh connections. Again, a host may be specified with the format <host>:<port> as with telnet, but there is not much use of this since ssh is a secure protocol. If you do not want an ssh port to be opened by the console, or you do not want to support ssh at all, you may not specify this property in the configuration.
You may prefer not to specify the telnet and/or ssh hosts/ports in advance. Instead, you can start the telnet/ssh servers with console commands from command line after you start the shell.
The new console supports configuration through the Equinox Configuration Admin service. If you want to configure it this way, you should programmatically register two configurations in the Configuration Admin service - for the telnet and ssh, respectively. The PIDs of the two configurations are osgi.console.telnet and osgi.console.ssh, respectively. Both configurations should contain the following properties:
enabled - if the telnet or ssh should be enabled by default. The values are true and falsehost - the host on which telnet/ssh connection should be acceptedport - the port on which telnet/ssh connection should be accepted
If you configure the console through the Configuration Admin service, you shoud not specify the properties osgi.console and osgi.console.ssh in the config.ini file. Instead, you should add the property osgi.console.useConfigAdmin=true. Also, you need to install the Configuration Admin service.
The last property, osgi.console.ssh.useDefaultSecureStorage, specifies what storage does the ssh use for storing sensitive data like user passwords. The ssh in the console uses JAAS base user authentication. The default login module, which comes with the console, uses password based access. You can provide a custom login module, but if you decide to use the default one, this property must have value true.
The file org.eclipse.equinox.console.authentication.config specifies which login modules should the JAAS framework use for user authentication. In our case in the file we specify the name of the default login module of the console. If you provide a custom login module, you should change the entry in this file. The name of the file could be arbitrary, as long as you correctly pass it as an argument when you start the java process.
Following are the arguments, which should be passed to the JVM:
logback.configurationFile specifies the location of the logback configuration file.ssh.server.keystore specifies the name of the file, where the ssh server will store its keys.org.eclipse.equinox.console.jaas.file specifies the name of the file, where the default JAAS login module will store user credentials. Credentials are encrypted. This property must be set if the default login module is used.java.security.auth.login.config specifies the name of the file, where the JAAS login modules are listed. In our case the value is org.eclipse.equinox.console.authentication.config
By default, when the new shell starts, it reads from standard input and writes to standards output, in addition to the telnet/ssh sessions support. If you want to start the shell non-interactively, that is the shell will not read and write to the standard streams, you should add to the argument gosh.args=--nointeractive to the VM arguments. In this case you still will be able to open telnet and/or ssh connections, if you have configured the shell accordingly.
A few more words about the logging bundles. In this blog the slf4j api and implementation from the Eclipse Orbit repository are used. Any slf4j implementation may be used instead. If you decide to use another slf4j implementation, you should also use the original slf4j api (instead of org.slf4j.api bundle from the Eclipse Orbit repository). You can take the original slf4j api and different implementations from here. If you decide to use any of them, do not forget to update the config.ini file by adding the new bundles and removing the old logging bundles. And also, do not forget to provide the correct logging configuration file, appropriate for the logger of you choice.
Console Support in Equinox
The Equinox Framework has always provided a console administration option. You just have to start it with -console option and you get the well known osgi> prompt on the command line. It comes with a nice set of commands for install/uninstall/start/stop of bundles, inspection of bundles and service, etc. So far, so good.
However, Equinox usually is not used standalone, but as an OSGi implementation in much more complex systems, for example the Virgo Web Server. In such cases the ability to administer the framework through remote connections becomes important. And when the remote access enters the game, the problems are not far away.
Equinox provides a very basic support for remote access - it just listens for network connections, takes the socket input as it is, and writes the output back to the socket. It is not very user-friendly, I admit. And I do not even mention about security.
In the last two versions of Equinox - 3.6 and 3.7 - a support for extending the built-in console was introduced. This allows you to extend it with usability features, but still nothing ready-to-use comes with the framework.
What is New in the New Shell
To overcome the limitations of the default Equinox console, a brand new project was started in the Equinox incubator, whose purpose is to provide much more usable and secure console. It is based on the Apache Felix Gogo shell and adds a number of features to it.
Following are some of the major features of the new console:
- telnet and ssh connectivity
- command line editing
- JAAS based user authentication (for the ssh)
- tab completion
The new shell is backwards compatible - all commands available in the built-in Equinox console are available in the new one too.
Running the Console
The Console in Virgo Web Server
The new console is currently adopted in Virgo Web Server 3.0.0. You can find how to configure it in the Virgo Web Server documentation.
Running the Console Standalone
The most impatient, or those who do not want to spend time putting the parts of the shell together, may prefer to download the archive with everything necessary to run the console. You just need to execute the startup.bat script in the console folder of the archive after you unzip it in order to run the shell.
Now for the more curious - the following steps describes how to put together the different parts of the shell.
Step 1. Download Apache Gogo Shell from Eclipse Orbit repository
Step 2. Download bundles for SSH support from Eclipse Orbit repository
Step 3. Download the new console implementation bundles:
Download Equinox incubator build and take form it the following two bundles (located in the plugins folder):
- org.eclipse.equinox.console.jaas.fragment
- org.eclipse.equinox.console.supportability
Step 4. Download logging api and implementation, required by the ssh bundles:
Step 5. Configuring the console.
- Place the above bundles in a folder. Add to this folder the Equinox itself
- Create a subfolder called configuration.
In configuration create a file config.ini
- config.ini file should have the following content:
osgi.bundles=./org.apache.felix.gogo.runtime_0.8.0.v201108120515.jar@start,\
./org.apache.felix.gogo.command_0.8.0.v201108120515.jar@start,\
./org.apache.felix.gogo.shell_0.8.0.v201108120515.jar@start,\
./org.slf4j.api_1.6.1.v20100831-0715.jar@start,\
./ch.qos.logback.core_0.9.27.v20110224-1110.jar@start,\
./ch.qos.logback.classic_0.9.27.v20110224-1110.jar@start,\
./ch.qos.logback.slf4j_0.9.27.v20110224-1110.jar,\
./org.apache.mina.core_2.0.2.v201108120515.jar@start,\
./org.apache.sshd.core_0.5.0.v201108120515.jar@start,\
./org.eclipse.equinox.console.supportability_1.0.0.N20110816-2000.jar@start,\
./org.eclipse.equinox.console.jaas.fragment_1.0.0.N20110816-2000.jar
osgi.console.enable.builtin=false
osgi.console=2223
osgi.console.ssh=2222
osgi.console.ssh.useDefaultSecureStorage=true
- Create a file with name org.eclipse.equinox.console.authentication.config in the configuration subfolder.
- Add the following entry to org.eclipse.equinox.console.authentication.config file:
equinox_console {
org.eclipse.equinox.console.jaas.SecureStorageLoginModule REQUIRED;
};
- In the configuration subfolder create logging configuration file called logback.xml. For now write in it
<configuration>
<root level="off" />
</configuration>
to turn off logging by the ssh server.
- Start the Equinox framework with the following command (you may put this in a startup.bat file to easily start it every time):
java -Dlogback.configurationFile=configuration/logback.xml
-Dssh.server.keystore=configuration/hostkey.ser \
-Dorg.eclipse.equinox.console.jaas.file=configuration/store \ -Djava.security.auth.login.config=configuration/org.eclipse.equinox.console.authentication.config \
-jar org.eclipse.osgi_3.8.0.N20110816-2000.jar
And voila! Now you should be able to access the console over telnet on port 2223 and over ssh on port 2222. The default username/password for ssh accdess are equinox/equinox. On the first login this user is deleted and you are prompted to create a new user.
Next time I will describe the different configuration options.
