Anyway, once I have started that box I have found a strange problem. Almost every 3rd keypress event was automatically repeated like 4-5 times. It was impossible to type the WiFi password First I have connected the USB keyboard to finish the setup and then I have started looking for the reason. My first idea was to replace the battery as I do not know how many months this unit has spent on the shelf at FutureShop. But that has not changed anything. Puzzled, I walked downstairs to get the remote from the first BoxeeBox and noticed that it was on. Which was not something I expected. So, it turned out that the remote from my new BoxeeBox was also waking up and controlling the one located nearby!

Turning off and removing the power plug from the first box has proven that the theory was correct. After that a quick search on the Internet has shown that, in fact, there is a way to pair the remote to BoxeeBox (I wonder why it is not paired at the factory). All you need for that is to make sure no other devices are on, put the remote next to the box and press and hold the OK button (the "select" button on the simple side of the remote, the one in the middle of the navigation keypad) for at least 5 seconds.

I have paired both remotes to each corresponding BoxeeBox and was happy to see that they now do not control anything but the device they are paired to!

Just imagine how customers may hate this produce if two people leaving in adjacent apartments buy it

The only problem is that even if you select "Registration: when needed" the device registers with SIP provider upon your first internet voice call and then keeps connected to it. Even if you disable WiFi - it will re-enable it in a few seconds and reconnect.  Nothing seems to stop it As result, the battery life of N8 phone will be typically around 4-5 hours even if you do not do anything else with your phone, making it pretty much useless brick.

However, there is an easy way to control this SIP account state. Make sure you define the destination for it first (in the profile settings). Then go to Contacts application and switch to the tab displaying the internet call icon. You will see your account there. You can click it and this will start the sign-in process. And once connected, you can click "Options" and "Sign out" there. You will see that the SIP connectivity icon will disappear instantly and most likely your WiFi connection will also close.

Of course while you are not registered with SIP server you cannot receive incoming calls. But it is typically not a problem as most of the people use it for outgoing calls only.

Happy battery saving!]]> http://myhowto.org/mobile/87-controlling-sip-connectivity-on-nokia-phone/feed/ http://myhowto.org/spam/84-morality-and-technological-progress-are-we-destroying-our-society-or-making-it-a-better-place/ http://myhowto.org/spam/84-morality-and-technological-progress-are-we-destroying-our-society-or-making-it-a-better-place/#comments Fri, 12 Aug 2011 02:07:48 +0000 nick http://myhowto.org/?p=84 Strange title for a technical site, isn’t it? This is something I was thinking about for a while. Not sure if I will be able to explain my thoughts but I am going to try. These thoughs were, in fact, triggered by the recent strike of Canada Post. This is not the place for me to talk about how do I like the unions and how they abuse the power they have. In fact, the reason for the strike was very simple: the crown corporation is not making enough money so it cannot afford to support the luxurious working conditions (well, it is all relative but…) for their staff.

While I do not want to discuss if Canada Post should go to the bank and borrow money to pay their employees for 6-week vacation, the point is - the mail system is not handling as many letters as before, this is why the revenue is down. And all this thanks to the technology. Even considering that not that many people are geek enough to do banking entirely online, the people are sending less and less letters replacing them with emails, chat, SMS etc.

What is about looking at this from the perspective of the postman who does not have enough work to do because of the technological progress? The technology breaks the business models and destroys the professions that were successful and in demand for many dozens (and sometimes hundreds) of years.

Another example from the same area - local utility company called Hydro Quebec. For many years the army of their employees was walking the streets of every single village in Quebec, going into each backyards (often not being very welcome there ) to personally read the numbers from the mechanical electric meters and write them down on the paper. Later they have switched to some kind of mobile terminals, I believe, but they still had to look at the needles and do some basic rounding of the numbers. Not always successfully, by the way Hydro Quebec was technically ready to replace the old meters with new ones that could be accessed over phone lines many years ago. Apparently, they have faced strong union resistance. Obviously, you do not need an army of human beings to read the meters in this scenario. Hydro Quebec has given up that time. Now they are back with wireless meters that allow to read it from a good distance in a second. As a consumer, this time I hope the progress will win. However, this does mean that the army of meter readers will have nothing to do and they most likely will have to look for another job.

Every time you buy online from that strange e-shop in Hong Kong which ships for free from another side of the planet you make the local guy selling these goods on nearby plaze very upset. Quite often I see articles praising the role of local business and small retail, blaming Wal-Mart and others for destroying the local economies and forcing the people out of business. Strangely enough, that guy at local plaze often buys his goods from the same store in Hong Kong, but after receiving them he or she increases the price ten times trying to make a decent profit. Is it moral to use the modern e-commerce to destroy the local business? Is it moral to heavily overcharge the consumer in modern e-commerce world?

I do believe that the technology, the “new ways of doing things” are killing the businesses and making some ordinary people who did not deserve more poor. But why does it happen? Is technology an evil? I do not think so. I think the problem is that the people are not flexible enough to adjust to the reality and instead of trying to find a suitable place in the new world they either give up or try to adjust the reality so it reshapes back to the model they like (yes, that is where the unions become very helpful ).

You may ask: what if the people cannot adjust? At the end, not everyone is capable of learning something new because of the educational background, age, personal learning skills, family situation…you name it. I think the answer to this is quite simple: the business should be more responsible for forseeing their future and helping the people to use their skills in new roles so the business can benefit from it even more. Letter traffic is down, not enough work for the postman? Well, e-commerce absolutely requires shipping packages from online stores and their warehouses to the customers. Reallocate your staff, put more emphasis on serving online business, make the rates attractive so they ship with Canada Post instead of UPS (which is now often cheaper that the regular Canada Post package!). You lose on letters - you gain on packages. High volume will bring in additional revenue and you will need your staff to handle it.

Unfortunately, this is not as simple as it sounds but one thing I know for sure - we advance very fast and the technology evolves making our lifes…well, at least different. I know it because I work with technology for many years and I am doing my part in pushing it forward. It can be slowed down but cannot be stopped and reversed. So the small guy running little local store selling HDMI cables for $60 will be out of business sooner or later. People need to look forward to see what is coming (and if they cannot  - they need help with that!) so they can find their rigthful place in ever-changing world.

In this article I am going to explain how to intercept and decrypt the SSL traffic between an Android phone and third-party server. This is what sometimes the people have to do to troubleshoot complex communication problems.

What is needed:

• Linux computer with following tools:
  • OpenSSL (http://www.openssl.org)
  • SoCat (http://www.dest-unreach.org/socat)
  • Java (J2SE, at least JRE 1.6 or later)
  • Bouncy Castle library (http://bouncycastle.org/download/bcprov-jdk16-141.jar) - place this JAR file into your .../jre/lib/ext directory
• Jailbroken Android phone. In fact, jailbreaking is needed only to install new trusted CA certificate
• Some patience

SSL connection are secured end-to-end. Intercepting SSL traffic without having the private key from the server it is going to is mostly pointless, unless you are planning to attach the crypto algorithm itself. And since the connection is going to a 3rd party server most likely you won't be able to get their private key (unless you steal it first ).

The goal is to terminate the SSL connection locally to have the date clear-text and then forward it to the regular destination establishing another SSL connection. Essentially, you want to break one end-to-end SSL connection into two independent segments and see what is in between. Optionally, you can simulate the real backend with some kind of fake server - if this is what you need for your debugging purposes. This does not change the method.

This is definitely a well-known "man-in-the-middle" technique. The goal of this article is to provide a summary of how it can be applied for a particular scenario.

2. The challenge

The applications rely on the platform to provide reliable and trust-worthy SSL tunnel (for HTTP transactions, for example). The platform does everything it can to make sure that the server certificate is valid by checking it against the preloaded trusted certificate database.

While it is possible to generate so-called self-signed certificate, in most of the cases it won't work. By default all SSL implementations refuse to accept self-signed certificates (best case - the application or browser may prompt you if you are willing to trust that untrusted certificate, but with the exception of Web browser it is almost never the case). The application has to do something specifically to allow SSL implementation to accept such a certificate. The whole goal of our exercise is to intercept the traffic from the application that is NOT designed to accept untrusted certificates (otherwise it is too easy!).

While you may already understand what are we going to do, you realize that getting another certificate to substitute the existing one that does not belong to you from Thawte will be problematic

So, what we will be doing is creating a new Certificate Authority (CA), creating new server certificate signed by it and loading the certificate of this CA onto the phone so it trusts it the same way as it trusts Thawte or others.

In addition to that, you need to pass the traffic through your machine so you can manipulate with it. There are several scenarios you may consider:

• Redirecting all traffic normally going through your 3G connection to your machine. This can be achieved by setting up a VPN connection to your host.
• Redirecting all WiFi traffic going through the network you control (your home network). In this case the VPN solution will also work but there may be easier wy to do it - you can always set your machine as default gateway if it is located on the same subnet as your phone. You can do it either directly on the phone or by modifying the configuration of your router if it allows you to mess with the routing table.
• USB cable connection. I personally did not try that but it should be relatively easy to use the same trick and set your machine's IP address ad default gateway for your phone.

In either case what is important is that you pass all the traffic from your phone through your machine. By the way, if you are manipulating the routing information on the phone, you can route only the traffic you need to your host, not everything.

Lets assume the phone has 3G data connection and your Linux box has a public IP address of 1.2.3.4. We will set up the VPN tunnel between the phone and your machine using 10.0.0.0/24 network, your phone will get 10.0.0.100 address an your interface on Linux side will be 10.0.0.1.

Lets assume the phone application you want to debug is connecting to 11.22.33.44 on port 443 (HTTPS). It may be easier (and more realistic) if you know the host name the application connects to instead of the IP address. You will have to determine it yourself. One hint I can give - use tcpdump and look at the DNS requests your phone is launching. If you see that it is looking for the host that corresponds to the service the application connects to - this is what you need. You can also reverse-engineer the application to find out if it contains something like a host name inside. It is also possible that the app receives the target host name from another server via its protocol - using some kind of service discovery mechanism. The possibilities are unlimited.

Why it is important? Because if you want to pretend to be "that" server you need to know how the SSL client (application) will be calling you. If it is calling by host name, it may get resolved in many different IP addresses for redundancy and load balancing purposes. You can issue a server certificate for an IP address, a host name or domain name (wildcard certificate). This is why I highly recommend to find out the host name the app is connecting to instead of the IP address as latter may change for each request.

We will be using VPN method. I personally used XL2TP package for Linux as I prefer L2TP over P2TP. You can get any kind of VPN server you want, it does not really matter. We won't be covering the VPN setup in this post.

Once you get your VPN up, your phone will use 10.0.0.1 as default gateway, will have ppp0 interface up with address 10.0.0.100. And your Linux host will also have ppp0 interface up with address 10.0.0.1. If you run tcpdump on it, you will see that all your phone's data traffic is now going through it. Part of the job is done!

Don't forget that if you want your phone to go to the Internet you need to enable IP forwarding and configure NAT on your Linux system. Both are done in two shell commands:

echo 1 > /proc/sys/net/ipv4/ip_forward
/sbin/iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

For NATing there may be other ways to achieve the same result, just look for HOWTO for iptables.

4. Lets get to the cryptic stuff

First, lets create our CA:

openssl genrsa -des3 -out ca.key 4096 

openssl req -new -x509 -days 365 -key ca.key -out ca.crt

One important note concerning the CN value for the CA certificate. Make sure it is different from any CNs of the server certificates you are going to generate later. Do not put the same server name there.

Now lets generate our server certificate. Lets assume we have an application that makes requests to http://myapp.mydomain.com. These two commands will generate the server key and Certificate Signing Request for it. Make sure you enter "myapp.mydomain.com" in the CN field of your CSR - this will identify the server!

openssl genrsa -des3 -out server.key 4096
openssl req -new -key server.key -out server.csr

Now your CA will have to carefully verify your information and charge you a fee for signing your server key. Since you happen to own this CA it is up to you if you want to accept the CSR or not As well as you decide how much do you pay to yourself for issuing the certificate

openssl x509 -req -days 365 -in server.csr -CA ca.crt 
-CAkey ca.key -set_serial 01 -out server.crt

If you want to get rid of the password (this is only for you and for your debugging purposes so why bother?):

openssl rsa -in server.key -out server.key.insecure
mv server.key server.key.secure
mv server.key.insecure server.key 

Great, now you have the server key, server certificate. The only problem is that Android will not consider you a trusted CA at this point as your CA certificate is not it its database. We need to get it on the device.

Android stores the CA certificates in /system/etc/security/cacerts.bks. Pull this file from the phone:

adb pull /system/etc/security/cacerts.bks cacerts.bks

Store a copy of the original file somewhere as you will eventually want to restore it.

Add your CA certificate to this database:

keytool -keystore cacerts.bks -storetype BKS -provider 
org.bouncycastle.jce.provider.BouncyCastleProvider -storepass changeit
-importcert -trustcacerts -alias CACERT -file ca.crt

Make sure you answer "yes" to the question "Trust this certificate?". Now you have updated your database. You need to push it back to the phone and for that your phone has to be jailbroken. I was not able to push the file directly in place here is what I was able to do:

• push modified cacerts.bks to /sdcard
• login to the shell (adb shell), become root (su -) and move the file into right place
• fix the owners and permissions: chown root.root cacerts.bks ; chmod 644 cacerts.bks
• if your filesystem is mounted read-only, you can remount it using "mount -o remount,rw /system" and then back "mount -o remount,ro /system". I did not have to do it on my device.

Now reboot the phone.

We want to intercept the TCP traffic going through our host to myapp.mydomain.com, port 443 (HTTPS). Again, there are different ways to do it with iptables on Linux - depending on how granular you want your configuration to be. For simplicity lets do this:

iptables -t nat -A PREROUTING -i ppp0  -p tcp --dport 443  
-j REDIRECT --to-port 4443

This will redirect all TCP traffic to port 443 to your local port 4443.

Now lets make sure we are ready. The best way to do it would be to hit your Linux box with the Web browser from Android phone and see if it complains about the invalid certificate or not. If it does not, the app will most likely won't do it too. Since we do not want to run the Web server, we will just simulate it with socat (greatest network debugging tool, by the way!):

socat OPENSSL-LISTEN:4443,reuseaddr,verify=0,cert=/tmp/server.crt,
key=/tmp/server.key,cafile=/tmp/ca.crt,debug  -

This will set up a listener on local port 4443 that uses SSL protocol, the given server key and certificate (the one you have generated for myapp.mydomain.com, remember?). What do we expect? We expect to see the HTTP request from your Web browser on your console (since the output for socat tool is specified as "-").

On Android (assuming your VPN is open and ready) open the browser and type "https://myapp.mydomain.com/blah-blah". Hit "Go". If you did everything right, your browser will request the DNS to resolve "myapp.mydomain.com" and will get back 11.22.33.44. Then it will open a TCP connection to 11.22.33.44, port 443. The connection will go from 10.0.0.100 address via 10.0.0.1 gateway. Instead of being NATed and sent out, it will be redirected to local port 4443 on your Linux machine because of the iptables rule you have created. And your socat will accept this connection. It will present the SSL certificate issued for "myapp.mydomain.com" which is signed by your own CA. On the device, the SSL implementation will find the appropriate CA certificate in cacerts.bks file and will determine that it is a trusted one. Since the server name you are connecting to (myapp.mydomain.com) matches the CN value of the certificate presented by socat (myapp.mydomain.com) and this certificate is properly signed by trusted CA (yours) the SSL implementation will consider the connection as trusted. As result, you shall see "GET /blah-blah HTTP/1.1..." on your Linux console. If this is the case, you have the working environment. If your browser fails to connect, then you have a routing or iptables problem. If your browser complains about untrusted certificate, you did something wrong when issuing the server certificate or loading CA certificate on the phone.

All you need now it to bridge your connection to the target. You can do something like this:

socat -v OPENSSL-LISTEN:4443,reuseaddr,verify=0,cert=/tmp/server.crt,
key=/tmp/server.key,cafile=/tmp/ca.crt,debug,fork OPENSSL:
myapp.mydomain.com:443

This command will accept the connection on port 4443, decrypt it, open the SSL connection to myapp.mydomain.com, port 443 and pass unencrypted data between these two sockets. ",fork" option will ensure that it is done for each incoming connection and "-v" option tells socat to display the traffic going back and forth.

This method can be used to debug most of SSL connections between Android phone and a 3rd party server without modifying the applications on the phone and causing any SSL handshake failures. While it may apear to be a hacking technique, the author does want to emphasize that the material is presented for educational purpose only and to perform end-to-end protocol debugging of your own software. Obviously, like in many other cases, the technically feasible method can be used for different purposes and the author is not responsible for any damage caused by it.

In this post I used some recipes from other people and appropriate credits and references are at the end of the post.

• instead of using Internet or WiFi connection the USB cable connection can be used
• Linux can be replaced with any Unix

1. Guide on making self-signed certificates
2. Importing certificates on Android phones
3. OpenSSL library
4. SoCat
5. xl2tpd

Once I tried it for the first time, I have found two problems. First of all, the picture was just too small. It was approximately half of the image produced by the projector. Unfortunately, PK301 refuses to stretch the image both vertically and horizontally at the same time unless you use VGA cable. Second problem was the audio. Small mono speaker available in the projector is just not good enough for a movie. N900 has great speakers but, unfortunately, the software is designed so it redirects the sound to the connected AV device once video output is connected..

After searching a bit on maemo.org I have found a method of configuring the parameters of the video signal produced by the projector. The following commands will fix them and N900 will use almost entire area of the projected image (you have to be root to run them):

	echo 26,10 > /sys/devices/platform/omapdss/overlay2/position
	echo 658,464 > /sys/devices/platform/omapdss/overlay2/output_size

First one moves the image to the top-left corner, the second one sets the maximum resolution I could get from the handset. The image uses the entire height but does not use the entire width. All you need to do to make it perfect is to tell the projector to stretch the image horizontally (only).

Now about the audio. N900 use pulseaudio and ALSA sound drivers. I did not have time to figure out how to alter the audio configuration to change the logic. Instead I have decided that all I need to do is to restore the same mixer settings as the phone uses when no AV cable is connected. With ALSA it is very easy to store the current settings to the file.

	/usr/sbin/alsactl -f ~/alsactl.bak store

Then the settings can be restored by the following command:

	/usr/sbin/alsactl -F -f ~/alsactl.bak restore

So what I have done - I have stored the original settings, then the settings active when AV cable is connected and combined them to produce a config that keeps TV out enabled but uses internal speakers instead of audio output via AV cable. The file is available here

Now I can enjoy the full-screen movie on the ceiling and the sound is played via N900's stereo speakers.

First of all, what is Web Analytics everyone is talking about these days? Briefly, Web Analytics is a set of methods for collecting, storing, analyzing and reporting of the Web traffic. The goal of Web Analytics is to optimize the traffic for achieving various (business) goals. It is important to mention that Web traffic is essentially what you generate when surfing through a Web site using one of the modern browsers. Web Analytics does not require the complete traffic information and sometime does not require traffic information at all. Think about it more like about the set of events associated with the user navigating through your Web site - "user X used the search page", then "user X added an item to the shopping cart", then "user X accessed the checkout page" and then the absence of traffic for this user would be interpreted as a bounce.

Thanks to the modern (and very old) Web standards, the Web traffic is very easy to describe. The only protocol used is HTTP (over TCP or TLS), the type of the content transferred to the user is primarily HTML. JavaScript running on the client side allows you to do complicated things on the user's side. A little bit of magic and you have a wonderful tool like Google Analytics that produces tons of useful information with minimal configuration efforts, for free and almost seamlessly for the end user. This tool will work for a e-commerce Web site and for the personal blog without any changes. We will not go in the details, I assume you are familiar with the technology - since you are reading this post.

Now we are approaching the most interesting part. Why the same methods do not work for them mobile content? What is the mobile content, first of all?

Unlike in the modern desktop world with the powerful personal computers connected to multi-megabit-per-second networks, the mobile world is quite different. First of all, not all mobile data traffic is HTTP-based. And not all data being transferred is HTML. Mobile world is full of rich applications using their own data formats (not always something more or less standard like XML-based), quite frequently compressed and often very application-specific. Even if HTTP is used as transport, the navigation is mostly controlled by the rich client application UI rather then by the links and buttons on the Web page. As result, while the application is downloading the avatar image for one of the Instant Messaging contacts the mobile user may be engaged in the chat with someone else. Just by looking at the data exchange between the application and the handset, even if you can decode the traffic, you will never able to figure out the navigation path.

It may be also difficult to track the unique users with the rich client applications. First of all, some services may be session-less or anonymous, just like in the Web world. Secondly, even if the service supports a notion of session, the association between the user ID and the session ID happens somewhere at the initial handshake and then the session ID is hidden somewhere in the protocol. Unlike in HTML data transferred via HTTP there is unlimited number of ways to embed the session ID in the protocol, it is application-specific. Thus, even while having complete access to the traffic between the handset and the service destination, the user tracking method has to be application protocol-specific.

It is also impossible in most of the cases to tag the data being transferred. Even if the rich client application is using HTTP as transport, the HTTP implementation is usually quite limited. It does not support the cookies (except the ones that the application requires itself). The application may not be able even to handle very basic HTTP features like redirects.

However it does worth mentioning that sometimes the mobile communications offer something you will not get in the Internet. For example, quite often when the HTTP traffic passes through WAP gateway, additional information about the device and the subscriber gets inserted in the HTTP headers. Sometimes it can be the phone number of the subscriber, but this usually has to be configured on the WAP gateway. However, since in Web Analytics we are interested primarily in detecting the unique users, any kind of value provided as "x-up-subno" header would work. The biggest issue with using these values is that they can be easily spoofed by someone. However, it is not worse than spoofing the HTTP cookies so I do not believe this is an issue.

Of course, significant part of the mobile traffic is generated by the mobile browsers. And I believe that with the time the portion of the traffic generated by the rich client applications will decrease, as we get better devices with high resolution screens and the browser software getting as close as possible to the PC. However, at this point there is a number of completely different mobile browsers: ranging from OpenWave browser that understands only WML and does not support even WML Script and up to real Firefox running on Maemo-based devices. Most of these browsers are not the same as ones we run on the PCs, even if they do support Javascript - it is not usually quite limited. The same applies to HTML, CSS and Flash. Even some basic things like HTTP Referrer may not be supported or this field may be filtered out. And if the extreme case - Opera Mini. When you access a Web site using Opera Mini, a server at Opera actually downloads the contents of the page for you, renders it and sends it to the handset in very compact OBML format. Due to this variety of the Web browsers it is quite often when the content providers host special versions of the Web sites optimized for the mobile users.

Due to these numerous browser issues the only method that will really work would be the server log parsing. And, you will be lucky if the cookies work for some of the users

Again, just like in case with the rich applications, there may be some interesting opportunities out there: additional HTTP headers or even the original client IP address (if provided) could be used to uniquly identify the users.

The following methods will most likely work for the mobile Web traffic:

• Image tags - if the chandset browser supports images (and they are not disabled because of the limited amount of traffic offered with the data plan!)
• Redirecting the links - modifying the links on your pages so they point to your analytics engine and then redirected to the real content

All of this being said, in the mobile world most of the standard data collection methods used for Web Analytics either do not work at all or are not reliable enough. Even if a method appears to be working, there is always a danger that it provides partial data and completely ignores entire user segment. This is not acceptable for Web Analytics (unlike regular data loss which is perfectly acceptable as long as it is not specific to a particular user segment).

Thus, I believe that it is quite hard or even impossible to come up with a solution similar (in terms of simplicity and portability) to Google Analytics for the mobile content, unless you are targeting mobile Web users with top-of-the-line handsets (and data plans). But if that is the case, there is not too many differences between the regular Web users and mobile Web users, except that the latter ones probably access special mobile version of the Web site.

There is definitely an interesting area related to the Web Analytics for rich client applications. It is quite obvious that any Analytics solution that targets this kind of traffic has to be either passive and highly customizable (to be application protocol-specific) or the mobile application has to be modified in order to collect the client-side events and pass this information to the Analytics server (just like instrumented Flash applications do).

First of all, why I am so interested in using Wi-Fi. I have a Blackberry without data plan and it was my decision not to buy one. I believe that the prices for mobile data access in Canada are just insane. I am talking about the real mobile data access, not unlimited browsing. At the end, what is the point in using Blackberry browser if you have the applications like Opera Mini? And you probably want to use GoogleTalk and some other applications that do not use HTTP via WAP Gateway. In addition to that, I have a mobile network around me most of the time and when I do not - I can live without the mobile data. And Wi-Fi access costs nothing. And I assure you - there are virtually no network applications that could not work over Wi-Fi (I saw the statements from some customer support "specialists" that "GoogleMail absolutely requires mobile data plan"!).

Unfortunately, Research In Motion has introduced the support for Wi-Fi as "side" components. Unlike for the regular data network access, it is not very well integrated with the applications using the configuration "by default" Here is what is takes to use Wi-Fi in any Blackberry application: you just need to add "interface=wifi" to the URL the application is trying to access. That's all. The device will use Wi-Fi network connection in order to access this URL. The same applies to TCP connections. Simple? Very simple.

However, for unknown reason RIM has not made one little thing that would change everything: a setting (per application or a global one) that would use Wi-Fi for all connections that do not have "interface=wifi" parameter. As result, any existing application that currently relies on the mobile data network has to be modified and rebuilt, although the modification is extremely simple.

Why was it done this way? Someone did not think about how convenient is to use Wi-Fi for any network connection by default when it it available? Or because RIM makes money on selling the phones to the operators and the operators make money on (over)charging the customers for the mobile data access? I hope that it was the first reason but I am afraid the second is more realistic.

Me and some of my friends have already tried to contact a couple of companies that provide free Blackberry applications for accessing their services (Google, MeteoMedia, Yahoo...) and explain the problem. Surprisingly, there were no responses at all.

I have started looking for a way to modify the binaries of the existing applications in order to force them to use Wi-Fi network connections. The biggest challenge is the proprietary binary data format used by RIM. However, for some applications it is relatively easy and I enjoy using GoogleMail and GoogleMaps on my 8120 purely over Wi-Fi. I will publish an article on how to achieve it as soon as I can.

First of all I would like to thank drbolsen for his efforts in reverse-engineering the COD file format. Although I find a bit strange that he publishes the information that is a bit incomplete and his COD decompiler application just does not work, still, great job!

Some background information: As far as I know, Research In Motion has 6 different signatures available: 3, RBB, RRT, RCR, RCC, RCI. First one is to sign the internal RIM code, RCC is a special signer for using the open-key cryptography from Certicom (once I tried to find out how much does it cost and the answer was $xx,xxx dollars!). RCI is an internal signer for Crypto API. 3 other signers are for mortal developers Typical applications use RBB, RRT and RCR signers only.

Lets open the COD file in your favorite hex editor. Scroll down to the end of the file. If the application is signed, you will see a section at the end that is easy to recognize (see the picture below):

First of all, you will see the signer names (in blue). You may see different number of them - depending on which parts of the API the application is using. If you see RCC there - abandon your hope (unless you have this signature from Certicom). If you see only RBB, RRT or RCR - proceed.

Find the first signer name (i.e. closest to the beginning of the file) and then look towards the beginning of the file for the following hex sequence: 01,00,84,00. This sequence seems to be always aligned to the 4-byte word so it is easy to find (red). Frequently it is preceded with zeros, most likely this is padding. Now all you need to do is to cut off everything starting with 01,00,84,00 and save it to your new unsigned COD file.

Now you can modify the COD file any way you want to. After that you need to re-apply the signature, obviously - your own signature. SignatureTool does not seem to analyze the code in order to figure out which signatures are required for your code. Instead, it relies on the .csl and .cso files generated by the RIM compiler (RAPC). The files are very simple and can be copied from any Blackberry application. So, if your COD file is called "myfile.cod", just create "myfile.cso" as follows:

33000000=RIMAPPSA2
52424200=RIM Blackberry Apps API
52435200=RIM Crypto API - RIM
52434300=RIM Crypto API - Certicom
52434900=RIM Crypto API - Internal

and "myfile.csl" as follows:

52525400=RIM Runtime API
52424200=RIM Blackberry Apps API

That's it, now SignatureTool will gladly sign your modified COD file.

References

1. COD file template

It is usually against my habits to write about the political or economical issues since this is a technical blog. However, I do make exceptions when there is certain overlap between the technical and political/economical sides of a problem that results in slowing down the technological progress. And this is the case, unfortunately. If you are Canadian, you will understand me, for sure.

How did it start: Government of Canada Opens Up Wireless Industry to More Competition

OTTAWA, May 27, 2008 — The Honourable Jim Prentice, Minister of Industry, today announced the opening of the bidding process for the Advanced Wireless Services (AWS) spectrum auction. This auction follows a commitment made by this government to enable more competition in the wireless market...."Our government's intentions are clear: to achieve lower prices, better service and more choice for consumers and business," said Minister Prentice. (taken from this article).

At the beginning it appeared to be a great idea. Here in Canada we definitely lack the competition. Especially when it is about the telecommunications. You want a cellphone? Yes, you have plenty of choices: you can pay $40-50 CAD to Rogers. Or, alternatively, you can pay the same amount to Bell. Telus? Oh, right, I heard about this operator too...Guess what - you will see the same price and the same conditions. Actually, these days the only reasonable choice is the operator with a GSM network. How many of them do we have? Exactly one Do you know how much does it cost for a Rogers customer to send an SMS from NYC to Montreal (600 km)? More than it would cost to send a regular letter!

Some of use were very frustrated when Rogers have made their iPhone offer. Want to learn more - go to this site and read the user messages there (warning: contains coarse language and viewer discretion is advised ). This iPhone offering perfectly demonstrates the position of Rogers on the development of wireless service in Canada.

So, back to the auction. It was a long and interesting process, with new players getting in, some players dropping off and some previously unknown names appearing from nowhere. But at the end...

Sold! 15 Companies Bid Almost $4.3 Billion for Licences for New Wireless Services

OTTAWA, July 21, 2008 — Consumers and businesses will be the winners in a more competitive wireless market. Industry Canada today announced that 282 licences have been conditionally assigned to 15 companies in the Auction of Spectrum Licences for Advanced Wireless Services and Other Spectrum in the 2 GHz Range...."The auction exceeded our expectations in terms of the level of competitive bidding activity. I hope the industry keeps this competitive spirit alive as it enhances and expands its services with improved access to the spectrum," said the Honourable Jim Prentice, Minister of Industry. "The industry now has an unprecedented opportunity — thanks to the government's Advanced Wireless Services (AWS) policy and auction — to develop products and services that offer choice to Canadian consumers and businesses. We think consumers will be the big winners in this auction." (see the full article here).

Now let look at the list of winners more closely. I will not quote the entire list, just first 3 entries:

Almost entire block A has been purchased by Rogers, block E and F - by TELUS and Bell.
The names look suspiciously familiar...But this is not the worst thing - at the end, on the 4th line of the list we actually see "9193-2962 Québec Inc" (AKA Quebecor Inc) nd "Globalive Wireless". These are the saviors of the Canadian consumer who were expected to change the rules of the game.

Auction winners and the Canadian geography

There is a little problem with these two winners - Globalive has bought the frequencies in every Canadian province but Quebec (except a tiny part in Northern Quebec). And Quebecor - nowhere but in Quebec. As result of this auction we have not got any potential national operator in Canada.

Quebecor/Videotron and their plans

Recently Quebecor has finally made a decision what are they going to do with the airwave rights they have acquired. Surprisingly, they are actually going to build an HSPA network!

Quebecor Inc. said it will be investing upwards of $800-million to build a high-speed wireless network to launch a cellphone service in Quebec by the end of 2009....He later added that bundling its cellphone service with its TV and Internet offerings will give the best value to its customers....Although there has been widespread speculation Quebecor would partner with fellow new entrant Globalive Communications Corp. to share the cost of building a network, Globalive's CEO Anthony Lacavera said no deal has been made between the two companies. (read the full article here.

Apparently Quebecor is not looking for the partnership with Globalive. Which means if you are a Videotron customer who has home phone, Internet and TV from them you may get a small rebate on your wireless service but once you drive for 200 km to the west - welcome to the roaming zone. As well if you come to visit Montreal from Toronto you will have to pay roaming fees just like if you are coming from another country.

It seems that the success of the auction is measured only by the amount of money raised by the participants. The primary goal (to provide the frequencies to a winning company that could become a national operator) has not been achieved. The winners are actually the existing operators who make the using of cellphones and the data service a luxury here in Canada.

References

1. Spectrum Management and Telecommunications
   

The basics

First of all, there are two data types in Oracle: DATE and TIMESTAMP. The primary difference between two of them is that TIMESTAMP supports higher time precision - up to 10^-9 seconds. DATE values can be created by parsing a string expression using a particular data format using TO_DATE function and converted back to the formatter string by using TO_CHAR function. For the TIMESTAMP values there is a function called TO_TIMESTAMP (similar to TO_DATE) and the same TO_CHAR function can be used to format timestamp as string.

There are also INTERVAL types used to hold the duration of the time period. We will not be talking about the intervals in this article.

Both DATE and TIMESTAMP values internally use the number of days since January 1, 4712 BC. This is also called "Julian" day number.

Simple date/time arithmetics

Oracle supports a number of arithmetic operations with DATE and TIMESTAMP values. For the adding and subtracting Oracle converts the dates into day numbers. Thus, adding something to a DATE means adding certain number of days to the date. The result is converted back to DATE or TIMESTAMP.

SQL> select to_char(sysdate, 'DD-MON-YYYY HH24:MI:SS') from dual;

TO_CHAR(SYSDATE,'DD-
--------------------
23-OCT-2008 01:20:22

SQL> select to_char(sysdate+2, 'DD-MON-YYYY HH24:MI:SS') from dual;

TO_CHAR(SYSDATE+2,'D
--------------------
25-OCT-2008 01:20:22

The number you add or subtract does not have to be integer, thus, you can easily do something like:

SQL> select to_char(sysdate + 1/24 + 3/(60*24),
   'DD-MON-YYYY HH24:MI:SS') from dual;

TO_CHAR(SYSDATE+1/24
--------------------
23-OCT-2008 02:23:22

In the example above we add one hour (1/24) and 3 minutes (3/24*60) to the current date.

Obviously, you cannot multiply or divide dates like numbers, this does not make sense.

Subtracting one date from another gives you the number of days between them (which, of course, can be a float number!):

SQL> select sysdate + 1/24 + 3/(60*24) - sysdate from dual;

SYSDATE+1/24+3/(60*24)-SYSDATE
------------------------------
			.04375

Converting between UNIX timestamp and Oracle DATE/TIMESTAMP

Since you already know that the date/time values are based on the number of days, you can probably imagine how one can calculate the UNIX timestamp from Oracle date. You can use something like this:

SQL> select (sysdate - to_date('1970-01-01 00:00:00',
    'YYYY-MM-DD HH24:MI:SS')) * 86400 unix_ts from dual;

   UNIX_TS
----------
1224725729

The similar method can be used to convert the UNIX timestamp back to DATE - you just need to add the right number of days to the epoch.

References

DATE Datatype