MyHowTo.org

J2ME/MIDP Programming For Mobile Phones Good, Bad and Ugly


Today there are not too many technology alternatives when it comes to creating software for the cellphones. There is a number of platforms out there (like Symbian, Linux and now even Apple with the iPhone) but there only 3 of them that can be really considered for mass-market development: J2ME (the most widely available one), BREW and Windows Mobile. Windows Mobile is limited to a very few cellphones, these are usually expensive PDA-like devices. BREW is available only for the CDMA phones, they are just starting penetrating the GSM market. But I would way that only J2ME was truly successful mobile platform so far. Most of the devices, even BREW or Symbian-based, support MIDP 1 or 2 as their primary application development platform. Only with MIDP you can develop applications without having to buy expensive SDKs and compilers and to deal with the complicated licensing issues.

So, from the commercial perspective J2ME was a success. How is about the engineering perspective? Is J2ME a good development platform or a source of constant headache for the software engineers? In this article I will try present my vision of the J2ME platforms. The article does not require knowledge of the J2ME technology, however the knowledge of Java platform in general will help to understand it better.

A little disclaimer before we start. In the beginning of my software developer career I worked for Sun Microsystems on various projects, including the very first version of KVM simulator, precursor of the Wireless Toolkit. It was a very exciting and challenging project and at that time I did not realize what</b> actually I am working on, it was just a project for one of Sun’s partners. After about 3 years I have started working for another company as senior software designer of the mobile applications. This position allowed me to look objectively at what happened with J2ME from the developer’s perspective.

  ### MIDP building blocks

Just a little note at the beginning: in this article I will be talking about MIDP 2, unless I explicitly say "MIDP 1" The current version is 2 so there is no need to go back to the previous one (although there are phone manufacturers who STILL create new cellphones with only MIDP 1 implementation available!).

The Java Platform

This picture represents an overview of the components of Java ME technology and how it relates to the other Java Technologies (image from "Java ME Platform Overview" page at Sun). We are talking about the second column from the right - the one designed for the "mobile phones and entry-level PDAs"

The core of the MIDP platform ("profile", to be more accurate) is the set of basic classes offered by CLDC configuration of the Java platform and the little virtual machine called KVM. Here is what you get as Java developer from CLDC:

  • java.lang (and java.lang.ref in CLDC 1.1) package which is a stripped-down version of the corresponding J2SE package, includes only the object data types supported by CLDC and minimal set of objects that are absolutely required (like Thread and System).
  • java.io package which is a very small (but complete) subset of J2SE version of the same package. Obviously, it does not include the things like BufferedReader etc.
  • java.util package is limited to 7 utility classes
  • javax.microedition.io package (that is J2ME-specific). This package provides protocol-specific entry-points to various I/O methods (which are limited by default to ContentConnection, DatagramConnection and StreamConnection)

That's all for CLDC. This is a really spartan library (69 classes and 15 interfaces) but combined with the power of Java programming language it is sufficient to implement the algorithms that are supposed to be implemented on a small device.

MIDP completes this library by adding the libraries required to develop the real applications, as it is shown on the picture below (taken from the same page at Sun):

CLDC Wireless Platform

MIDP offers the following functionality (in addition to what CLDC offers):

  • Adds a couple of new connection types to javax.microedition.io package: HttpConnection, HttpsConnection, SocketConnection, ServerSocketConnection and the support for secure stream connections.
  • Adds the Timer and TimerTask classes to java.util package.
  • javax.microedition.midlet package containing the parent class for all MIDP applications (MIDlets)
  • javax.microedition.pki package to support the security certificates
  • javax.microedition.rms package offering a simple persistent storage mechanism
  • javax.microedition.media and javax.microedition.media.control packages (part of Mobile Media API) for manipulating the media resources
  • javax.microedition.lcdui and javax.microedition.lcdui.game packages offering the user interface APIs

It comes to 107 classes and 43 interfaces in total. It is still quite spartan library but it is complete</i>, i.e. theoretically you can build any application just using it.

  ### JSRs for everything you can imagine

As it was described in the previous chapter, MIDP provides the complete runtime environment, i.e. you can write almost anything that can run on a mobile device using it. However, the environment imposes certain limitations such as:

  • Limited application size does not allow to write too complicated code or to carry substantial amounts of data with the application. So, if for whatever reason you want your mobile application to use Corba most likely you will waste most of the allowed application space just implementing it :)
  • It is absolutely impossible to talk to any native library from a MIDP application. Unlike J2SE there is no public native interface like JNI. Actually, there is an interface called KNI, but it is absolutely useless for the application developers, you need to be able to recompile the whole KVM to be able to use it. And even if there was a native interface, it would be impossible to provide any kind of native code since the underlying platform is usually something unique and 100% proprietary.
  • MIDP provides very limited access to the platform resources. For example, if the phone support a feature and there is no Java API available for it, there is absolutely no way you can access it, even if you know how to use it from the native code

Sun has made a very smart move starting the Java Community Process (JCP). There is about 300+ open specifications (read - APIs) available through this process, approved by most of the big software vendors and offering the standard way of accessing various hardware features, protocols, data etc. Some of these JSRs are applicable for the J2ME platform. Just to give you an example:

  • JSR-118 - MIDP version 2 itself
  • JSR-135 - Mobile Media API, allows the MIDP applications to work with audio and video, from playing a simple tone sequence to capturing a video stream from the on-board camera
  • JSR-179 - Location API for J2ME
  • JSR-180 - SIP API for J2ME
  • JSR-184 - Mobile 3D graphics API for J2ME
  • etc..

So, the actual mobile platform for J2ME developer is MIDP plus a number of additional libraries (some of them are mandatory parts of MIDP 2 itself). For example, if you need the application to use Bluetooth and your device supports JSR-82 - great, discovering the services on the remote device and using the OBEX protocol is a matter of a couple of hundreds of lines of code. OK, now you are about to ask the question - what if JSR-82 is not available?

  ### Theory and practice

Until now everything appeared to be clear and reasonable. Simple but complete Java-based runtime, a number of nice libraries designed for the mobile applications…But the reality is not that perfect, unfortunately.

First of all, there is a lot</b> of the things in MIDP that are clearly marked "implementation-specific". The API documentation contains quite a lot of "implementation may…", "implementation should", "depends on the implementation" etc. Too much for two hundreds of classes.

Is it bad, you may ask? May be not. J2SE specification contains enough of these statements too. But think about how many different implementations of J2SE you have to deal with? I bet you have J2SE from Sun installed on your machine. How many of them exist? Just a couple. And you always have luxury of developing your application just for one particular JVM. This is not the case for the mobile phones. You have to deal with more than a dozen of JVM vendors who have several different builds of their JVMs and in every build the developers had different ideas about how to handle these "implementation-specific" details.

Just to give you an example. javax.microedition.lcdui.Form class. I like this class, it is very easy to build a UI form and prompt the user for some input. However, lets look more carefully at the documentation…

  • "The implementation handles layout, traversal, and scrolling.". Yes, it sometimes does. One some phones the label is displayed above the input field. On some phones it is displayed next to the field. One some phones the first letter of the label gets converted to upper case. There is no way you can control the font used in the form, even if your phone supports multiple fonts. Traversal…On some phones you can move from the bottom-most item to the top-most by pressing "down" One some phones you cannot. I prefer not to think about how different is the behavior or the Form and the CustomItem when it comes to calling the traverse() method. Scrolling, especially when combined with the resizeable input fields (or the fields that scroll horizontally) sometimes almost made me crying :) I think I could continue but you have got the idea.
  • "Implementations may provide additional conditions under which a row break occurs". Some JVM developers read it as "implementations do not have to provide any clearly defined conditions under which a row break occurs".
  • "If the number of items does not fit on the screen, the implementation may choose to make it scrollable or to fold some components so that a separate screen appears when the element is edited.". On some of the Sony Ericsson there is no inline editing available for the Form-based screens at all. For every field you need to click on it, it brings up a new full-screen editor, you have to use it and then to press a button to go back to the original screen.
  • "The implementation calls this method when the available area of the Displayable has been changed". Yes, sometimes it does. But it does not always pass the values that make sense :(
  • The whole command handling business is a big mess. There is no way you can have full control on the appearance of your menus.

Believe me, I could continue with the Form but it is not my goal to complain about all the problems I have (and have solved one way or another) in the past. Because of all these problems, there is no way you can build a consistent user interface for your application. Usually the Form implementation does not allow you to build you the screen that works just like all other screens outside of your application and it also does not allow you to build a screen that works in the same way on all the handsets. Lose-lose situation.

Some of these problems are probably not that important unless you have really strict UI requirements. However, sometimes it is not about the external UI requirements, it is about common sense and the usability of your application.

Why? Seriously, I do not know. Everyone seems to be "certified" by Sun, every JVM implementation was apparently tested with thousands of the tests and all of them went flawlessly. There is no explanation why the PushRegistry has fundamental problems on half of the existing phones, why using HTTP implementations invent some headers and corrupt the ones the developers specifies? Why the phone that is sold in millions crashes when you access the network in the background while entering some text into the form?

OK, the situation with the MIDP API itself is more or less clear. What is about these functionality-reach JSRs? Lets take a look at the list of additional JSRs available on one of the most popular Motorola V3 phone (GSM):

  • JSR-82 - Bluetooth API
  • JSR-135 - Mobile Media API
  • JSR-205 - Wireless Messaging API

This is ALL. No JSR-186 and 187 (Presence and Instant Messaging), no 3D API, no JSR-75 (filesystem and the address book access), JSR-135 does not allow you to take pictures with your camera, etc, etc. Most of the phones offer very limited set of JSRs and these JSRs (their specs are also full of "implementation-specific" things) are not implemented in full. For example, most of the existing phones that have the ability to store the files on the phone (pictures, ringtones etc) do not offer the Java application any way to access them (via JSR-75).

Again, why? Why the manufactures do not implement these nice additional APIs so you could build the location-aware applications capable of displaying 3D images and have a voice interface? I cannot say for sure but I will tell you what I am thinking about it. This also applies to the question why do most of the KVMs are barely suitable for developing the business applications.

The problem is that probably around 90% of all mobile applications are games. Everyone makes them, lot of people buy them, operators love reselling them and getting their share. They do not really want to mess with the modern data-based services, it is must better to charge the ridiculous fees per minute for the voice calls. MIDP games are using only about 70-80% of MIDP API, some of JSR-135 and that's all. Why the implementations of the PushRegistry are all broken? Because there are no games using them! There are no games that send and receive SMS. Very few games need the network, that's why the HTTP and socket implementations have dozens of various problems. Why the Forms are full of bugs and inconsistent with the phone's interface? Because the games do not use Forms. And so on.

  ### Partial solutions

Some of Java-friendly device manufacturers have realized that just supporting MIDP 2 with a bunch of additional JSRs is not enough to give the developers the full power to develop good applications for their handhelds. One of them is Research In Motion. RIM offers an alternative (and superior) API that does much more than MIDP. For example:

  • You can really develop the applications that look exactly like other applications on this handheld
  • You have more control on the networking, graphics, application lifecycle, resource management etc
  • There is a working way of signing your 3rd party application so it can get access to the sensitive resources like network, address book, email etc. Signing MIDP applications is technically easy, however, using the 3rd party certificates is usually useless, you get almost the same permissions as unsigned applications.

However, RIM APIs are not perfect. They have enough bugs, they have certain parts that are not properly documented, some of the APIs are not consistent and they are not always backward-compatible. But you do not really have a choice on this device. It was a good attempt, but it will remain a proprietary API that will never be ported to any other platforms.

There is a number of other attempts to reinvent the wheel and do what Sun has partially failed to do. For example WIPI, the mobile platform (Wireless Internet Platform for Interoperability) developed in South Korea. They have actually managed to come up with quite advanced platform and there are APIs for C and Java. However, it is available mostly in Korea so unless you target this part of the market it is not an option to consider.

Some smart guys from Germany were really tired with porting the MIDP applications to various handsets. Since they realized that they will not change the world they live in, they have decided to write a nice porting layer on top of MIDP. It is called J2ME Polish. Basically, this is an API that in some way duplicates and in some way extends MIDP API and provides more consistent platform for the mobile applications. It is highly customizable through the build scripts and the styles. The consistent behavior comes at certain price. For example, they simulate the forms using Canvas. In order to do it, they have to draw everything, including the flashing cursor. Thus, you have to carry certain amount of J2ME Polish code with your application and the more features you use, the bigger is the library (can be a couple of dozens of Kb). However, the approach is quite popular, there is a lot of companies using J2ME Polish and a number of mobile software companies had to come up with something similar to it for internal use.

  ### Conclusion

So, finally, is MIDP a good mobile platform? The problem is that it is the only mobile platform that is really widely available and that offers at least certain level of the application portability - thanks to Java. But the portability is far from being perfect. And, even if we put aside the portability issues that are, in fact, usually due to the really low-quality implementations, there is still a problem. The mobile platforms develop faster than the development platforms. MIDP applications sometimes are just not good enough for the modern phones. Of course, it is usually about the UI and "look & feel", however even the fundamental things like the application lifecycle creates the problems. Most of the modern phones have the operating systems that execute multiple applications at the same time. MIDP does not have any kind of background state for the applications (pauses is a different state). Plus, limited platform integration does not allow the MIDP applications to use most of the features offered by the handhelds.

It seems to me that Sun was looking only for one year in future when designing the MIDP platform and did not think about what will happen in 5-7 years. They probably overestimated the speed of the adaptation of the new technologies, they did not expect that MIDP 2 will live so long and that CDC with Personal Profile will not be available on the mass-market devices. As result, I would say, the developers are stuck with a technology that is still not mature and at the same time is already too old to support the modern handhelds. It definitely endangers the position of Java on the mobile market.

  ### References

  1. Java ME Technology
  2. Java Community Process
  3. Wireless Internet Platform for Interoperability
  4. J2ME Polish
blog comments powered by Disqus

Published

11 September 2007

Category

j2me

Tags