So, my new blog can be found at http://techandsp.blogspot.com/.

Matt Porter’s Android Mythbusters presentation and Harald Welte’s reaction

I haven’t seen the presentation live, but I had a look at the slides. Impressing work done by Matt putting all this information together. However, we all knew that Android only (ab-)uses Linux, without making use of the GNU userland for a long time, didn’t we?

In his presentation Matt has shown things such as Android’s udev “replacement” that uses hardcoded values for device node creation and (on his blog) Harald has then come up with a statement I have found to be very strong:

The presentation shows how Google has simply thrown 5-10 years of Linux userspace evolution into the trashcan and re-implemented it partially for no reason. Things like hard-coded device lists/permissions in object code rather than config files, the lack of support for hot-plugging devices (udev), the lack of kernel headers. A libc that throws away System V IPC that every unix/Linux software developer takes for granted. The lack of complete POSIX threads. I could continue this list, but hey, you should read those slides. now!

Now both of these statements target technical details, but the root of the problem seems to be elsewhere.

Where is my Android 2.0?

Okay, that heading might not be making any sense in the context of this post at a first glance, but let me elaborate on that. Google and the Open Handset Alliance refer to Android as being an “Open Source” operating system, but the project is different from “real” Free Software projects: development takes place in a closed group and the results are shared with the community later on, when they are deemed to be ready.

This means that innovation also takes place behind closed curtains and that the community is not involved in the actual development process at all. Lately we have seen the result of that, as Motorola is bragging about working close with Google on Android 2.0 (”Eclair”), but the AOSP source trees, open for everyone to have a look at, show no signs of version 2.0. In fact no changes that might even remotely suggest the release of a new major version have been made public in the past few weeks. So where is the openess there?
Actually, the Motorola Droid has already shipped with Eclair on 6th, but still, there is no indication that Eclair will be made available to the broader public.

In short Android seems to be developed behind closed curtains, with hardly (read no) community input whatsoever and is sometimes released as Free Software, not what I would describe as an open development process.

The Android Market problem

As we have seen in the past Google is enforcing their copyright on proprietary applications that ship with pretty much every Android device, such as the Android Market. This has become really clear when Steve Kondik received  a cease and desist letter when packing the Google-proprietary applications into his ROMs. Okay, it’s Google’s right to enforce their copyright and there is nothing wrong with actually doing so, the thing I really have a problem with is something else: the Market is proprietary.

Now what this means should become rather clear. You can have an Android device without Google’s proprietary bits, but with default settings you just do not have any way of installing additional software. In my opinion the Market should be freed by Google themselves, or the community has to react and come up with a free replacement to overcome the vendor lock-in. Oh, you might know a replacement called SlideMe (or Mobentoo) already. Well, that bugger is proprietary too, so not a solution at all.

Nokia and Maemo to the rescue

In most discussions about the openness of Android someone throws in Nokia and Maemo, as a solution to the dilemma. Reading all those positive comments I simply had to give it a try, but all my hopes were destroyed within a few minutes.

Let’s start with the good news and let alone the reason why my hopes were destroyed for another minute or two. Maemo is based on Debian GNU/Linux and various Free Software components, such as GTK+, gstreamer, esd and friends. Most of the system is Free Software which is a good thing(tm) and reading all of this really got me into Maemo. Okay, some applications seem to be proprietary, but I am sure that could be fixed rather easily, so I could once for all use a truly open phone.

…and then came the SDK installer shell script:

#!/bin/sh
# Copyright (C) 2006-2009 Nokia Corporation
#
# This is proprietary software owned by Nokia Corporation.
#
# Contact: Maemo Integration <integration@maemo.org>
# Version: $Revision: 1110 $

Now there is one question you should ask yourself: Why would someone trying to promote his platform as being open make the *installer* script for its SDK proprietary? Come on, it’s an installer script, how much of your secret juice could be in there? What’s the problem with people modifying it and working on this installer script in an open development environment?

I had high hopes for Nokia actually doing a bit better than Google, but it seems they’ve failed to do so. It may be me overreacting, but a proprietary SDK installer shell script scares me enough not to install the SDK and have a look at it for now nor to think about buying a Maemo-based device in the near future. Please Nokia, either get the facts straight or provide us with a free SDK to your free & open platform.

So, in short, Google is bad at working with the community and creating a truly open development process, and Nokia simply fails in terms of not scaring off prospective developers for their open platform with the proprietary SDK installer. Do you have any solutions in terms of an open phone environment, apart from what OpenMoko has come up with?

More details on this topic will follow in the next few days.

UPDATE:

I have now integrated AdSense into this blog. As promised the front page does not contain any ads, but all other pages do. Ads are shown as a widget so they are not in-text and thus should not disturb you whilst reading.

So I guessed the lighttpd process itself must be experiencing problems of some sort, but after doing a “netstat -nat” I  knew what was going on:

tcp6       1      1 83.65.62.72:80          61.135.190.248:12474    LAST_ACK
tcp6       1      1 83.65.62.72:80          61.135.190.234:39671    LAST_ACK
tcp6       1      1 83.65.62.72:80          61.135.190.253:39211    LAST_ACK
tcp6       1      1 83.65.62.72:80          61.135.190.234:55160    LAST_ACK
tcp6       1      1 83.65.62.72:80          61.135.190.230:25836    LAST_ACK
tcp6       1      1 83.65.62.72:80          61.135.190.231:16865    LAST_ACK
tcp6       1      1 83.65.62.72:80          61.135.190.232:24266    LAST_ACK
tcp6       1      1 83.65.62.72:80          61.135.190.240:38441    LAST_ACK
tcp6       1      1 83.65.62.72:80          61.135.190.243:17726    LAST_ACK
tcp6       1      1 83.65.62.72:80          61.135.190.241:38206    LAST_ACK
tcp6       1      1 83.65.62.72:80          61.135.190.251:23892    LAST_ACK
tcp6       1      1 83.65.62.72:80          61.135.190.225:29675    LAST_ACK

Plus “a few” more of those. Now I’m not entirely sure whether it’s just some systems misbehaving or actually an attack, but my feelings told me this could have been intentional after all.
I did a quick whois on one of those IP addresses and came up with the 61.135.0.0/16 network which is owned by China Network Communications Group Corporation.

As the connections were made from pretty much every host in that network I had two choices: sit it out or block it.

I came to the conclusion that blocking the entire subnet from connecting to this system, at least temporarily, might be a viable solution and so I did.
However, afterwards I am asking myself whether I really had to block an entire 16-Bit network, so I am asking you: how do you handle such situations usually?

I am currently working on a (still private) project that uses Python’s unittest module and the underlying framework. Even though unittest is a great utility for creating unit tests I found that the output it generates is unusable for me. I wanted something different though. Maybe a bit more aesthetic than the simple command line output unittest provides.

So I started off writing a class extending unittest.TestResult to fit my needs. I soon realized that interfacing with this part of unittest is not as easy as it could be, but I still continued to write my class.
After two hours of hacking I noticed that this class had become a monster. It was huge and I felt uncomfortable having such a huge class lying around somewhere in a “runtests.py” file for the only reason of having that pretty output.

This was the point when I decided to move all that code into a separate project and try to come up with a more intuitive API. This was the second when sptest was born, about 5 hours ago.

What I did come up with is a small Python module that makes customizing the way unit test results are presented (or stored) easier. It currently includes two output handler classes. One providing fancy CLI output on ANSI terminals and the other one providing XML output.

Additional output handler classes could store the result of the unit tests in a database or send it to a central point on the network, but implementing that is up to someone else, for now.

Running unit tests with sptest is as simple as calling:

sptest.TestMain(TestSuite).run()

By default the FancyCLIOutput handler class will be invoked and you will see why the handler is called the way it is immediatly.

In order to generate an XML file containing the test results one just has to modify the call to sptest to look like this:

sptest.TestMain(TestSuite, output_class=sptest.output.XMLOutput).run()

sptest also provides support for preparation and cleanup functions. The only thing you have to do is define these functions and adjust the arguments passed to TestMain accordingly.

Most of the code is already documented and a doxygen configuration file for generating the html documentation comes with the code. Also, two examples are included that show how to use sptest.

It’s not like I got tired of blogging. The reason why there hasn’t been an update for such a long time is that I was doing my final exams in the past two months.

After passing my exams on Friday I should have time to write some articles again, so watch out for new articles here.

There are two main reasons for this step. First of all I am not able to open these files and I believe I will not be able to do so and get them properly rendered anytime soon. Secondly, people using the new Microsoft Office suite seem to be ignorant enough to think everyone is able to view those files, which is not the case.

I am trying to make one point here:
People sending emails to other people should always send files in internationally standardized formats (open formats), such as ODF or PDF, so that everyone is able to open them and use the attachments. Also, I am trying to make people sending out emails in those formats aware of the fact that not everyone can open them, not everyone wants to invest a lot of money in new applications and that some people generally prefer Free Software and that there is no way of using those files using Free Software right now.

Enough for the introduction, I wanted to explain how to achieve this behavior using Exim4:

deny message = Message contains attachment of unwanted type ($found_extension)
demime = docx:pptx:xlsx

Putting this snippet in the acl_check_content section of your exim4.conf should do the trick.

Oh, and while I am at it, you can easily use this snippet to drop mails with other attachments, based on the file extension.
For example, in order to reject all mails containing WMV files just use demime = wmv.

Note that this snippets checks for a specified file extension instead of a MIME type. People still can get mails through in those formats if they modify the file extension, so do not use this method as a security measure.

To sum the problem I was experiencing up: My anti-spam system (namely Spamassassin) did not detect spam mails anymore.

Now here is the reason it did not: After some more investigation of the problem I noticed that spam emails were received via a local connection (forwarded from fetchmail). However, one of my Exim ACLs says not to scan emails from localhost for spam.

So, the solution might be a hack, but it worked out perfectly. Starting fetchmail with the -S <servername> argument causes it to send emails to the given SMTP server rather than localhost. Using the real hostname of my server caused the “do not scan local mails” not to kick in and all mails received via fetchmail to be scanned again.

Problem fixed.

Now that my mail server is configured to do sender verification and a few very old mails came from domains or systems which are non-existent today about 50 mails ended up being frozen.

But how to remove all frozen mails from Exim’s queue? I ended up using mailq | grep frozen to get a list of all messages (and more importantly their message IDs) and saved that to a file. I then wrote a minimalistic Python script attached to this article to delete all those messages. Consider the script a quick and dirty hack, but it might come in handy for some of you. Get it here.