Planet Netfilter

Harald Welte: Motorola announces "Ming" phone with Android

Thu, 02 Sep 2010 02:00:00 +0000

For those who don't know: The Motorola Ming was the A1200, a commercially very successful Linux-based phone in China and other parts of Asia, using the EZX software platform, i.e. the kind of hardware that we once built the OpenEZX software.

Motorola has recently announced that they will follow-up with some android based ming phones. It is my suspicion that apart from some mechanical design aspects, those phones will not resemble the ming in any way, neither on the baseband hardware side, nor on the application processor side, and particularly not on the software side.

So it's probably nothing than a marketing coup, trying to connect to successes of the past. Not interesting from the OpenEZX point of view, I guess.

Harald Welte: More GPL enforcement work again.. and a very surreal but important case

Wed, 01 Sep 2010 02:00:00 +0000

In recent days and weeks, I'm doing a bit more work on the gpl-violations.org project than during the last months and years. I wouldn't say that I'm happy about that, but well, somebody has to do it :/

Right now I'm facing what I'd consider the most outrageous case that I've been involved so far: A manufacturer of Linux-based embedded devices (no, I will not name the company) really has the guts to go in front of court and sue another company for modifying the firmware on those devices. More specifically, the only modifications to program code are on the GPL licensed parts of the software. None of the proprietary userspace programs are touched! None of the proprietary programs are ever distributed either.

If that manufacturer would succeed with such a lawsuit, it would create some very nasty precedent and jeopardize the freedom of users of Linux-based embedded devices. It would be a direct blow against projects that provide "homebrew" software for embedded devices, such as OpenWRT and many others.

I've seen many weird claims and legal strategies when it comes to companies trying to deprive developers of their freedom to modify and run modified versions of Free Software. But this is definitely so weird that I still feel like I'm in a bad dream. This can't be real. It feels to surreal.

It's a pity that I cannot speak up more about the specific company in question right now. I'm desperately looking forward to the point in time where I can speak up and speak out about what has been happening behind the scenes.

David Miller: How GRO works

Mon, 30 Aug 2010 20:46:00 +0000

All modern device drivers should be doing two things, first they should use NAPI for interrupt mitigation plus simpler mutual exclusion (all RX code paths run in software interrupt context just like TX), and use the GRO NAPI interfaces for feeding packets into the network stack.

Like just about anything else in the networking, GRO is all about increasing performance. The idea is that we can accumulate consequetive packets (based upon protocol specific sequence number checks etc.) into one huge packet. Then process the whole group as one packet object. (in Network Algorithmics this would be principle P2c, Shift computation in time, Share expenses, batch)

GRO help significantly on everyday systems, but it helps even more strongly on machines making use of virtualization since bridging streams of packets is very common and GRO batching decreases the number of switching operations.

Each NAPI instance maintains a list of GRO packets we are trying to accumulate to, called napi->gro_list. The GRO layer dispatches to the network layer protocol that the packet is for. Each network layer that supports GRO implements both a ptype->gro_receive and a ptype->gro_complete method.

->gro_receive attempts to match the incoming skb with ones that have already been queued onto the ->gro_list At this time, the IP and TCP headers are popped from the front of the packets (from GRO's perspective, that actual normal skb packet header pointers are left alone). Also, the GRO'ability state of all packets in the GRO list and the new incoming SKB are updated.

Once we've committed to receiving a GRO skb, we invoke the ->gro_complete method. It is at this point that we make the collection of individual packets look truly like one huge one. Checksums are updated, as are various private GSO state flags in the head 'skb' given to the network stack.

We do not try to accumulate GRO packets infinitely. At the end of a NAPI poll quantum, we force flush the GRO packet list.

For ipv4 TCP there are various criteria for GRO matching.

Source and destination address must match
TOS and protocol fields must be the same
Source and destination ports must match

Certain events cause the current GRO bunch to get flushed out. For example:

ID field not being in sequence with existing packets
Don't fragment bit clear
TCP CWR congestion indication being set
TCP ACK sequence mis-match
Any TCP option mis-match
TCP sequence not being in-order

The most important attribute of GRO is that it preserves the received packets in their entirety, such that if we don't actually receive the packets locally (for example we want to bridge or route them) they can be perfectly and accurately reconstituted to the transmit path. This is because none of the packet headers are modified (they are entirely preserved) and since GRO requires completely regular packet streams for merging, the packet boundary points are known precisely as well. The GRO merged packet can be completely unraveled and it will mimmick exactly the incoming packet sequence.

GRO mainly the work of Herbert Xu. Various driver authors and others helped him tune and optimize the implementation.

David Miller: Converting sk_buff to list_head.

Thu, 26 Aug 2010 23:40:00 +0000

I've been trying to make this happen, off and on, for at least two years now. Most of the kernel is straightforward and uses the skb_*() interfaces we have for manipulating skb objects on a list.

So for those, simply tweaking the interfaces in skbuff.h will make them all "just work".

However there are a few other spots in the kernel which manipulate the SKB list pointers directly:

SKB fragment lists have a head of skb_shinfo(skb)->frag_list of the head skb and use only skb->next for linkage.
The GRO handling uses both ->prev and ->next with a single pointer head at napi_info->gro_list
Both ISDN PPP and the generic PPP code have a fragmentation handling engine which manipulates the SKB list pointers directly. I've very nearly converted the ISDN side to use the standard skbuff.h list interfaces, but it added regressions and I had to eventually revert.
The socket backlog handling uses a by-hand coded FIFO tail queue list of SKBs.

I'm taking another stab at this, and hopefully I can work out these wrinkles. It'd be a really nice change because of lot of uses of "struct sk_buff_head" which don't care about the spinlock or the packet count can be converted to simply "list_head" saving serious space in various datastructures.

Harald Welte: Convert RSS feed subscriptions from N810 feed reader to Android com.meecal.feedreader

Wed, 25 Aug 2010 02:00:00 +0000

I'm subscribed to a considerable number of RSS feeds, and so far I actually used to read them all on my Nokia N810, which is more or less permanently located at the bedside table

Now I wanted to import all the subscriptions into an Android RSS feed reader on the Galaxy S. Unfortunately the feed reader that I found most useable doesn't have OPML import. However, looking at its sqlite3 database for feed subscriptions, it was pretty easy to come up with a small perl script to generate "INSERT" statements for all the feeds from the N810 OPML file. In case anyone is interested, the script is available from here.

If you have any suggestions on a good Android RSS reader that can manage large number of subscriptions and put them into a tree/hierarchy of groups, feel free to let me know.

Harald Welte: India jails activist doing research on weak voting machine security

Sun, 22 Aug 2010 02:00:00 +0000

According to several sources such as indianevm.com, Hari Prasad was being arrested. He is part of a team of IT security researchers that gathered evidence to demonstrate how incredibly weak the security of India's voting machines is. For more details, read the indianevm.com article linked above, and the various quotes/links in it.

This is very upsetting. They should jail those who have authorized the deployment of such an insecure system in the first place. Those are the people responsible - not some researchers who go out of their way to uncover the technical problems to warn the general public about the inherent risks of this technology.

I sincerely hope that the authorities will understand the grave mistake they're doing here. Don't shoot the messenger. It's not his fault that engineer, engineering management and/or regulatory government authorities have permitted such a system in the first place.

Harald Welte: Started to play with the Galaxy S (GT-I9000) phone

Sat, 21 Aug 2010 02:00:00 +0000

For many years I'm on a more or less consistent hunt for finding a reasonably open and free mobile phone. This started in 2004 with OpenEZX, has continued with Openmoko, project gnufiish and has resulted in a bit of peeking and poking in the Palm Pre. However, none of those projects ever had the success I was hoping for:

OpenEZX was never really finished, and only for the 1st generation phones (A780) by the time they were long end of life
OpenMoko Neo1973 and FreeRunner were a great project, and they are still the most open+free mobile phones that ever existed. However, they're GPRS only and the hardware is even more outdated now then it was when we created it.
gnufiish was an attempt of running software from the Openmoko days (such as freesmartphone.org) on some E-TEN glofiish phones. However, we never could make the SPI-based modem communication work from our re-engineered Linux driver :(
Palm Pre is an interesting device, in that Palm provides easy root access, does not attempt to lock the device down with cryptographic signatures and provides full recovery flashing tools by means of WebOS Doctor. But once again, the proprietary communication protocol with the 3G Modem was the big blocker item for using real custom software and not the WebOS stuff they ship.

So I've constantly been on the watch for new devices that are coming out. Most of the phones you can buy in recent years are either running proprietary software like Windows Mobile, Symbian, Apples iPhone-OSX - or they run Android but then use some integrated Qualcomm Smartphone-on-a-chip product. The problem with the latter (from a Free Software point of view) is that Qualcomm is very secretive about their products, does not provide any kind of public documentation, and the ever-increasing integration between application processor and baseband processor makes it more difficult to run custom software on them.

The Samsung Galaxy S (GT-I9000) seemed like a good candidate to me, for several reasons:

Samsung does not use cryptographic signature techniques and gaining root as well as flashing the AP software is relatively easy
The phone is based on a traditional separate application processor (AP) and baseband processor (BP) design. The AP is a Samsung S5PC110, the BP is some Qualcomm MSM6xxx.
High-end hardware, with the S5PC110 running at 1GHz and 512MB RAM
Samsung provides excellent "GPL source code offers" containing the Linux kernel used in their firmware - including detailed instructions in how to build it. Also, many of the drivers are included under GPL, such as drivers for all the integrated peripherals of the SoC, some custom components like the USB multiplexor ASIC, etc. as well as the driver for the dual-ported RAM between the AP and BP for the 3G Modem communication
The Android RIL shipped by Samsung contains lots of debugging/decoding/dumping code that can make reverse engineering the AP/BP protocol.

So right now I'm in the exploration phase, making myself familiar with the bootloader, the flashing process, the userspace ABI of the custom (GPL licensed) kernel drivers, etc. It's a fairly pleasant experience so far, and I now have a debootstrap'ed Debian lenny on an additional ext2 partition on the SD card. This provides me with an actually useful userland I can chroot() into, such as lsof, strace, ltrace, tcpdump, etc. to do some more exploration of the phone.

The only real ugliness on the software side so far is the use of proprietary Samsung filesystems (RFS/TFS4). The only reason those filesystems existed, as far as I can tell, was to run legacy filesystems like FAT on top of raw NAND or OneNAND flash. This is mainly necessary if you want to export e.g. a FAT partition via USB Mass Storage to a Windows PC. However, the GT-I9000 doesn't have any OneNAND, but only an internal moviNAND (basically a SD-Card in a BGA package that you can solder on the board). MMC/SD cards already include the wear leveling algorithm, so there is absolutely no point (from what I can tell) in running the RFS/TFS4 stack.

In fact, in several forums people are complaining about the slow I/O performance of the Galaxy S, and they have a much better performance when using ext2/ext3 directly on that moviNAND device.

Harald Welte: Doing RFID related research and development again

Tue, 17 Aug 2010 02:00:00 +0000

More or less a bit surprising to me, I got again involved in RFID research, on which I hadn't really done much ever since my involvement in the OpenPCD and OpenPICC projects some five-to-four years ago.

It's a lot of fun, and I didn't seem to forget much. What really bothers me a bit is that the OpenPCD / librfid / OpenPCD integration never really was completed, and that libnfc doesn't work with OpenPCD. Let's hope I'll somehow find some time to change this. It just feels wrong that OpenPCD was the first hardware project created to encourage (security) research into RFID, and now all the current tools only run on the Proxmark or on proprietary readers...

Harald Welte: Worlds first 20 minute voice call from a Free Software GSM stack on a phone

Sat, 14 Aug 2010 02:00:00 +0000

As Dieter Spaar has pointed out in a mailing list post on the OsmocomBB developer list, he has managed to get a first alpha version of TCH (Traffic Channel) code released, supporting the FR and EFR GSM codecs.

What this means in human readable language: He can actually make voice calls from a mobile phone that runs the Free Software OsmocomBB GSM stack on its baseband processor. This is a major milestone in the history of our project.

While Dieter has been working on the Layer1 TCH support and the setup of the voiceband path in the analog baseband chip (audio ADC/DAC), Andreas Eversberg has been quietly working on getting call control of Layer3 into a state where it can do all the signalling required for mobile-originated and mobile-terminated call.

Combining both of their work together, they have been able to make a 20 minute long voice call from a baseband processor running a Free Software GSM stack. For all we know, it is the first time anything remotely like this has been done using community-developed Free Software. Five years ago I would have thought it's impossible to pull this off with a small team of volunteers. I'm very happy to see that I was wrong, and we actually could do it. With less than half a dozen of developers, in less than nine months of unpaid, spare-time work.

Sure, the next weeks and months will be spent on bringing the code from alpha level to something more stable, fixing known issues and known bugs, etc. But I'm confident the biggest part of the work on the OsmocomBB stack is behind us. Big thanks to the developer team driving this project forward.

Rusty Russell: fcntl lock starvation and TDB

Fri, 13 Aug 2010 04:09:03 +0000

The Trivial DataBase (ccan variant here) uses fcntl locks for consistency: records are chained off a fixed-size hash table (or the free list), and a 1-byte fcntl lock at the offset of the chain head protects all records in that chain.

There’s also a tdb_lockall() function which grabs a lock across all the hash chains at once to let you do arbitrary atomic munging. This works because fcntl() locks have an offset and length: you can lock arbitrary byte ranges.

Unfortunately, tdb_lockall() is subject to starvation, at least under Linux. This is because the kernel merely checks whether a lock is available and gets it if it can, rather than queuing behind someone else who wants a superset of the lock. So single byte lockers come in and out while the whole-file locker waits for them all to go away.

I’m not sure this is wrong, and as it’s simpler than the alternative, I’m not prepared to change it just yet. Fortunately, there’s a way of avoiding this starvation in userspace, which occurred independently to both Volker Lendecke and me. I called this variant tdb_lockall_gradual(), in which we try to lock each chain one at a time so we compete with the single-chain lockers on fair terms. My first naive thought was to try to lock all the chains one at a time in order, nonblocking, then go back and retry (blocking) any we failed to get. This is slow, and can deadlock against another process doing the same thing. Volker’s suggestion was much nicer: we do a non-blocking lock, and if that fails we divide and conquer. If we get down to a single record, we do a blocking lock.

I wrote a test program which fires off N children, each of which grabs a random chain lock for 50-150 milliseconds before sleeping for 1 second, then repeating. The parent waits for a second, then tries to do a tdb_lockall() or tdb_lockall_gradual() depending on the commandline. Running it five times and showing the average wait time for the lockall gives this:

Now, regarding performance. While there can be 10000 hash chains, this isn’t as bad as it sounds. The fast case is the uncontended one, and that’s as fast as before, and the contended case is already slow. I annotated the source to print out how many blocking/nonblocking locks it’s actually doing. Inevitably, if there’s contention, it will end up dividing down to a blocking lock, so log(numchains) locks before doing a blocking lock.

Processes	Blocking locks	Nonblocking locks	Seconds
5	0-2	1-27	0.03
50	8-12	93-111	0.20
500	13-21	130-170	0.29
5000	309-347	1660-1832	9.1

Sure, that’s a lot of locking when we’re competing with 5000 processes, but it’s less the naive one per chain, and it’s clear that it’s not the cause of the slowdown (we’re doing fewer locks per second than the 5 processes case).

And anyway, locking the entire database cannot be a speed-critical operation. Indeed, after the evidence here, I followed Volker’s suggestion to simply replace tdb_lockall() with the new implementation.

Harald Welte: Wondermedia WM8505 Linux + u-boot source code

Fri, 13 Aug 2010 02:00:00 +0000

In recent months, a number of alleged GPL-violation reports regarding products (tablet computers, mini netbooks and the like) using the Wondermedia WM850x line of ARM SoCs. People have been contacting me, as I was working as VIA Open Source Liaison, and there is the general belief that VIA and Wondermedia Technology (WMT) are one company.

I had investigated this issue even before there were any reports, and I'd like to publicly state that:

Wondermedia is a separate company from VIA, with independent management, making their own business decisions. The 850x SoC development was started inside VIA, but is no longer part of VIA for a long time.
Any references to VIA in the source code or old data sheets date from that time before the SoC business became part of Wondermedia
I have had assurances from Wondermedia, even before there were any allegations, that similar to VIA they explicitly notify their customers about the GPL and always provide their SDK / BSP as full corresponding source code.
Effectively, this means that GPLv2 Section "3a" is used. WMT has provided the Linux and u-boot source code to its customers, and thus has no obligation under GPLv2 Section "3b" to provide it to anybody else (any 3rd party)
So, if you buy a product including a WMT SoC and u-boot/Linux, like always, GPL compliance of what has been shipped to you has to be assured by the manufacturer of the product, not the semiconductor maker

Notwithstanding all of the above, Wondermedia was willing to provide the Linux kernel and u-boot source code of their SDK to me, so I can share it with the community. As indicated, they're not legally required to do this and I'm happy they do it anyway to show their good intentions.

You can download the released source code from the gpl-devices.org ftp-server, more specifically here are the latest Linux kernel (modified 2.6.29 android derivative) and u-boot source code archives.

This software is provided without any kind of support. If you see some GPL related legal problems (i.e. you believe it is incomplete), don't hesitate to contact me. To the best of my knowledge WMT (basically a small hardware start-up with small software development team) has no resources to actively push any of this mainline.

Rusty Russell: Bob Jenkins and lookup8.c

Thu, 12 Aug 2010 07:15:14 +0000

I pulled Bob Jenkins’ superlative lookup3.c into ccan some time ago, and with the work on TDB2 (aka 64-bit TDB) I wondered if there was a 64-bit variant available. I didn’t look hard enough: I should have checked his top-level page and seen lookup8.c before wasting his time with an email :(

I did note in my mail that since lookup3.c can return a pair of 32 bit numbers, I could combine those to get a 64 bit hash “but I wondered if it was possible to do better…”

His one-line reply was that he was working on a 128-bit hash now. Respectful and informative.

The gold standard test of online manners is how you respond to a dumb random email. A truly gracious gentleman/lady will respond as if you are far smarter far than you appear. Thanks Bob. And 128-bit; I’m a little stunned…

Harald Welte: Working on a document on smartphone hardware architecture

Sun, 08 Aug 2010 02:00:00 +0000

I've started to write upe some information on modern smartphone hardware architecture. It will be in a similar style to what I previously wrote on feature phones and gsm modem hardware, but with a specific focus on smpartphones, their multiple processors, memory sharing, AP/BP interface, audio architecture, etc.

I should have done this a long time ago. In fact, I think I should write more documents like that on various technical subjects. If you want to learn about low-level aspects of modern telephones, there is way too little published information out there.

Harald Welte: On my way to Taiwan for COSCUP

Sat, 07 Aug 2010 02:00:00 +0000

Tomorrow early morning I'll be on my way to Tapei/Taiwan. The main reason for this trip is the invitation to speak at COSCUP 2010.

I'm really looking forward to getting back Taipei, which has become something like my second home during the years I was working on Openmoko. I've really gotten used to life in this super-urban Asian metropolis... to the extent that I'm almost a bit homesick while I'm actually at home in Berlin/Germany.

Harald Welte: Playing more with Erlang

Wed, 04 Aug 2010 02:00:00 +0000

Last year I started to occasionally play with Erlang. People who know me as die-hard C coder who tries to avoid C++, Java and Python wherever possible will probably be surprised here now.

I have no intention of changing my general position on programming languages. I don't feel comfortable using something where I don't know and/or understand the immediate impact on how this code will be executed on the actual silicon.

However, if you have a need to play with anything that uses ASN.1, but particularly the aligned/unaligned PER encoding variants, then it is pretty clear that there is nothing available as Free Software that can compare to the Erlang asn1ct/asn1rt modules.

At that time last year I was doing some rapid prototyping with the RANAP protocol, and the progress was quite quick. I never had time to return to that project, so it (and my Erlang skills) were left dormant.

In recent weeks, I have picked Erlang up again - again to work on ASN.1 encoded messages: This time TCAP and MAP. While we still need the in-progress TCAP+MAP implementation in C for OsmoSGSN, there are other tasks at hand where an Erlang-based implementation might yield a much higher productivity.

So right now I'm working on a program that parses/decodes and iterates through every MAP component in a TCAP message and replaces certain fields, re-encodes the entire message and sends it off the wire. Once that is done, I think I'll actually try to do a more complete TCAP server and implement a simplistic HLR for OsmoSGSN testing.

Harald Welte: Official wiki page on GSMTAP created

Wed, 04 Aug 2010 02:00:00 +0000

I've come up with GSMTAP about two years ago while working on airprobe. The goal was to have something similar to what radiotap does in the wifi world: A pseudo-header that adds additional information and context that is not present in the actual message.

Initially, GSMTAP was intended to be a separate link-layer type in the pcap file format, but this would preclude its use in real-time protocol analysis. So I modified it to be encapsulated in UDP packets, which are sent and received using normal UDP/IP sockets.

Over recent years, GSMTAP has not only been integrated into multiple programs of the airprobe project, but is also understood by wireshark. OpenBTS has also decided to adopt the format and can generate GSMTAP messages for debugging purposes.

After creating OsmocomBB, it was taught how to generate GSMTAP messages very quickly, too.

So by now, at least when it comes to Free Software, it is definitely the de-facto standard for capturing/transmitting and analyzing protocol messages from the GSM air interface.

However, until now, there has never been any official "homepage" of the GSMTAP header. This has changed now, the GSMTAP homepage is now part of the OsmocomBB wiki.

Harald Welte: On the recent news items about the homebrew IMSI-catcher for 1500 USD

Sun, 01 Aug 2010 02:00:00 +0000

Some news sites seem to do very limited research and present it as big news that you can now build an IMSI-Catcher for a budget of USD 1500, using OpenBTS and a URSP.

Let me bring some clarity into this situation:

Fundamentally, an IMSI-Catcher is nothing special but a GSM base station (BTS) that is configured to the network country code (NCC) and mobile network code (MNC) of a commercial network operator.
In GSM, the phone has no way to authenticate and thus verify the legitimacy of the mobile network. This is like a "rogue access point" in a open (unencrypted/unauthenticated) WiFi network.
Thus, anyone who has a device that can run as a GSM base station has the ability to run an IMSI catcher.
There are two Free Software / Open Source projects for running your own GSM network, both have first been published in 2008: OpenBTS and OpenBSC.
None of those two projects are intended to be used as an IMSI-Catcher but for legitimate operation of GSM networks. However, if a user choses to configure the NCC and MNC of a commercial operator and allow "unknown/unregistered/unprovisioned IMSIs (SIMs) on his network, he will effectively have an IMSI catcher.
Such operation is in violation of spectrum usage regulations, even if you have a valid test/experimental license, since that license does not permit you to use somebody else's NCC/MNC.
Furthermore, such operation is in violation of criminal law in most jurisdictions. In Germany there is a separate offense in the criminal code, called Paragraph 317 Stoerung von Telekommunikationsanlagen, combined with Paragraph 202b Abfangen von Daten.
Furthermore, there are certainly civil claims to be made by the affected operator (and its subscriber) against anyone who unlawfully operates such a fake base station
OpenBTS and OpenBSC, as well as the problems resulting from this fake base station attack have been covered in a variety of conference presentations from 2008 through today.
Thus, there is nothing new about what has been presented at Defcon 18

Also, the theoretic basics ow how to operate an IMSI catcher are nothing new either. There are even a number of patents covering IMSI catchers, the first that I know of has been patented by Rohde & Schwarz in 2003. Also, see this blog post by OpenBTS founder David Burgess on this topic.

So all that you always needed is a bit of hardware and software to send radio waves containing messages formatted in the way how they are described in the (equally public) GSM specifications as published by ETSI and 3GPP. Commercial, proprietary systems have existed for a decade. From 2008 on, there is some Free / Open Source Software to operate GSM networks. The situation remains unchanged in 2010.

So please, remember this the next time somebody is trying to tell you that this is the latest invention since sliced bread.

Harald Welte: Dieter Spaar has started a blog

Sat, 31 Jul 2010 02:00:00 +0000

Dieter Spaar, who has been involved in various ways with both OpenBSC and OsmocomBB has just started a blog. This is good news and I hope this way he will get a bit more (much deserved) exposure on his great work.

Harald Welte: GSM Denial of Service by flooding BTS with RACH requests

Sat, 31 Jul 2010 02:00:00 +0000

At Blackhat US 2010, there was a Talk that (among other things) apparently included the subject of a RACH DoS on GSM base stations, implemented using my Layer1 of the OsmocomBB software.

As some news sites are covering this as "news": This vulnerability has been long known in the field and was - to the best of my knowledge - first demonstrated to a public audience by Dieter Spaar at the Deepsec 2009 conference in November 2009. You can get his slides.

The difficult part for many years has not been to know about the possibility of this weakness. Anyone who has read the GSM air interface specification will inevitably see that there is a limited number of RACH slots and a limited number of dedicated channels. Once you fill more RACH slots than the cell has dedicated channels, and you keep re-filling them at a higher rate than the cell can expire those dedicated channels, you have a DoS.

So rather, the difficult part was to implement it in practise, as traditionally all GSM baseband chipsets have been extremely closed, just like the very software (firmware) running on them. Today, starting from Q2/2010, it is very easy to do a proof-of-concept implementation, as we have created OsmocomBB: An Open Source baseband firmware.

Dieter Spaar's implementation predates OsmocomBB development by the better part of a year. At that time, he had to resort to binary-patching existing proprietary (binary-only) baseband firmware. So I think people should recognize his effort in doing the first practical implementation of that attack.

Harald Welte: A real-world practical A5/1 attack using airprobe and Kraken

Fri, 30 Jul 2010 02:00:00 +0000

At Blackhat USA 2010, Karsten Nohl has been presenting on a practical real-world A5/1 cracking attack. For recent years, Karsten, myself and others have been speaking at various opportunities, indicating that a practical attack using readily-available information and tools from the Internet is very possible, and that it is only a matter of time for somebody actually does it.

While Karsten has focused on the actual cryptographic attack, I've been putting in some time in projects like airprobe (a GSM receiver/decoder).

Now finally, a team of friends at the new Security Research Labs (founded by Karsten) in Berlin has put the pieces of the puzzle together.

Airprobe has been extended to fully support decoding of TCH/F (FACCH, SACCH and traffic), as well as SDCCH/SACCH control channels, and to specify the timeslot and physical channel configuration from the command line. Using this, you can

decode the AGCH, wait for an IMMEDIATE ASSIGNMENT of a SDCCH
decode that very SDCCH and wait until encryption is turned on
dump an encrypted burst where you have sufficient known plaintext
use a different program to actually recover the A5/1 ciphering key
feed that key into airprobe and decrypt+decode the ASSIGNMENT COMMAND of the TCH
use airprobe to decrypt+decode that assigned TCH/F

The external program to recover the A5/1 ciphering key is called Kraken and is also available from the SRLabs website.

So what are the limitations? Well, so far this only works on non-hopping cells with a single ARFCN. The limitations are those of the receiver hardware (and SDR software), and not really limitations of the airprobe GSM decoder or the actual software tools.

In the past I would have assumed that non-hopping and/or single-ARFCN cells are rare, but in fact we can find them even inside a big city like Berlin, from at least two of the four German GSM operators. So that's why this attack is very practical, no matter what the GSMA might say.

Harald Welte: I'm still alive ;)

Thu, 29 Jul 2010 02:00:00 +0000

In case you're wondering why there is such a long period with no updates: I've been travelling over the last week and barely had sufficient time to follow my e-mail and get the most high-priority work done. Hope to update the blog soon.

Rusty Russell: On Barriers To Entry

Sat, 24 Jul 2010 03:00:58 +0000

My philosophy for Free/Open Source Software comes down to this: that others can take what I do and do something awesome with it. Since I don’t know what you’ll need, I hand you every capability I have to get you started. Others granted me that very power to get where I am, so I seek to increment that.

It’s not always simple to do: sometimes you build it, and nobody comes. My experience is that clear code, convenient install, documentation and useful functionality all help. An encouraging welcome helps even more, as does getting the software out in front of people. It’s all about avoiding or lowering barriers.

We accept some barriers to modification which are reasonable: if you modify the software, I don’t have to support it. If I make you sign a support deal which says I won’t support the software (even unmodified versions) if you modify or redistribute the software, that’s not reasonable. If you get fired for publishing pre-release modified copyleft source code, that’s reasonable. If you get fired for publishing post-release, that’s not.

The hardware and electricity costs are barriers, but they’re undiscriminating and reasonable, so we accept them. Even the GPL explicitly allows you to charge for costs to make a copy. The software cost of the system is also explicitly allowed as a barrier. The software costs of the build environment are also accepted barriers (though historic for most of us): again even the GPL doesn’t require your software to be buildable with freely available tools.

As this shows, your choice of licensing is among your arsenal in keeping barriers low for co-hackers (who are not necessarily contributors!). I believe copyright gives too much power to the copyright holder, but as copyright is so hard to unmake, I favor the GPL. It tries to use legal power to meet the aims in the first paragraph: to force you to hand onwards all the pieces I handed to you. It’s also well understood by people and that common understanding gels a community.

Yet, as I alluded at the top, there are a realm of barriers which licenses don’t even try to address: the code could be an unmodifiable tangle, the documentation awful, the installation awkward, or the trademarks invasive. A license can’t make coders welcome newcomers, be friendly to upstream, responsive to bugs, write useful code or speak english.

The spectrum of barriers goes roughly from “that’s cool” through “I’m not so comfortable with that” to “that’s not Free/Open Source”. It’s entirely up to your definition of reasonableness; only in the simplest cases will that be the same point at which the license is violated, even if that license is explicitly drafted to defend that freeness!

So, don’t start with an analysis of license clauses. Start with “is that OK?”. Is there a fundamental or only a cosmetic problem? If it’s not OK, ask “does it matter?”. Is it effecting many people, is it setting a bad example, is it harming the project’s future, is it causing consternation among existing developers? If it is, then it’s time to look at the license to see if you can do anything. Remember that the license is merely a piece of text. It can’t stop anything, it can only give you leverage to do so. It certainly can’t make using the law easy or convenient, or even worthwhile pursuing.

To close, I will leave the last word to Kim Weatherall, who once told me: “if you’re spending your time on legal issues, you’re doing it wrong”.

Harald Welte: More musings on locked-down mobile phones

Sat, 17 Jul 2010 02:00:00 +0000

In recent days, the story about Motorola locking out its users (and developers) from their more recent Droid phones has made big news. As it seems, the exact functionality implemented by eFuses remains unclear, and the behavior of Motorola might thus not be too different from what has more or less become the industry standard.

For those of you who are not following the mobile world as close on a technical level as people like me do: In the last five years, more and more cellphone manufacturers have used cryptographic code signing to lock-down the software that you can run on the phone. Major parts of the system including the software update mechanism and the bootloader on the device contain a verification process of those cryptographic signatures to ensure that you can only software signed by the phone manufacturer.

I have seen this with the MotoMAGX phones like the ROKR2 v8, various Windows Mobile handhelds from HTC, The non-developer (non-ADP) version of the Google/Android G1 and many other phones.

This puts the user into a strange situation where he buys some hardware from the manufacturer, but yet doesn't have control over what this device does. Just imagine buying a computer, but being limited to run Windows 98 and Office 97 on it. You could not update to a later version of the operating system, and you could not install an alternative operating system such as a version of GNU/Linux. If the computer vendor decides that he will drop support for it, you will not even be able to install security updates to the operating system.

From my point of view, this is an abusive, anti-competitive behavior by the manufacturer. For no reason but his ever-growing hunger for power he makes you completely dependent on his decision. It is not in the control of the user, what operating system or even applications you can install. It is under the control of the manufacturer.

I would accept this if the phone was rented. In this case, I would only pay a small rental fee, but the phone is the property of the manufacturer and I am only using it. But the manufacturer actually sells the device. He wants to be paid the full price, but still not actually hand control over to the buyer.

Compare this with buying a CD-player that has arbitrary restrictions so it would only play CDs from one of the major music labels/distributors like EMI, but not CDs from any of the other publishers, for no technical reason whatsoever. Or buying a TV set that is locked down so you can only watch one TV channel, while you need to buy another TV for a different channel.

I actually think the antitrust authorities should investigate this behavior of the mobile phone industry. Simply compare it with the PC situation and look at the fact how often Microsoft has been judged in some kind of anti-competitive behavior in the PC world. In the mobile phone industry, the situation is worse than it ever was in the PC world, yet we do not see big antitrust cases being brought forward.

And please don't buy those pseudo-arguments that this has any relation to regulatory/FCC approval or the safety of mobile networks themselves. The entire software stack interacting with the mobile network runs on a separate processor (the baseband processor) anyway. It doesn't matter what you install on the application processor. Once again, compare it to laptops: You can insert a 3G miniPCI, expressCard or USB dongle. Inside this dongle you run the communications stack on a processor that is completely different from your main processor that runs your regular OS (be it GNU/Linux, OS X, Windows, Solaris or whatever makes you happy).

Harald Welte: Motorola locking down the DroidX and Droid2 in a nasty way

Fri, 16 Jul 2010 02:00:00 +0000

There are plenty of reports in recent days about the level of locking-down that Motorola is apparently doing on their most recent Android products, the Droid 2 and the Droid X.

This goes as far as to an (I believe unconfirmed) slashdot.org report claiming that not only there is the more or less typical DRM on software (i.e. cryptographic signature validation chain), but there also is an eFuse that that is blown if something happens wrong during the booting process.

To the best of my knowledge (and I'm doing mobile phone reverse engineering for about 6 years now), this is the first time I hear of something like this. If true, it sounds pretty dangerous to me. What if something goes wrong during an update (such as a power failure during software update)? What if you really have a non-correctable multi-bit error in your NAND Flash? In that case, cryptographic verification of the firmware fails and the eFuse would be blown, resulting in your device being a brick. This could eventually backfire massively to Motorola.

The best comment from the slashdot.org thread:
You can legally buy a gun that only shoots in the direction of the person pulling the trigger, but it doesn't mean it's a good idea.

Reading something like this almost makes me very depressed. Motorola is benefitting from the billions-of-dollar-worth development of existing Free Software projects like the Linux kernel, but they now want to take away the fundamental right to run modified versions of that very software. Somebody needs to slap them with a very large trout.

I'm not really surprised that they are doing it, though. Motorola has shown that direction even years ago when they first used SELinux as part of their later pre-Android Linux phones (EZX and MAGX). They didn't use it to enhance the security of the user, but to enhance the security _from_ the user.

Please also note this great post by Bradley M. Kuhn on the subject matter. If you don't know Bradley, he's been doing GPL enforcement for the last 12 years - for the Free Software Foundation and the Software Freedom Law Center. In his post, he actually thanks Motorola to publicly state that they actually want to lock their phones down (as opposed to Apple).

What's even more interesting though is his elaboration on the scripts to control compilation and installation clause of GPLv2. This is indeed something that most people tend to overlook when it comes to GPL[v2] compliance and we see this a lot during our gpl-violations.org work.

And in fact, for a very long time, I have been teaching and educating this fact during my GPL related talks and trainings: In software specific for embedded devices, the scripts to control installation are incomplete, if you do not provide a means to install the software onto the actual device. Where else would you be reasonably install the Linux kernel image that is made specifically to work on such a particular mobile phone model? Due to the custom nature of Linux kernels for embedded targets, it wouldn't even run anywhere else.

I've never taken any such issue to court so far - but it was a frequent dispute in out-of-court GPL enforcement we've been doing at gpl-violations.org. I'm definitely curious to see what will be the first court case addressing that issue. The ever power-hungry manufacturers of mobile phones seem like they deserve it.

UPDATE:
Apparently Motorola has released some statement that denies they use eFuses to brick the device. All it does is to render the device unable to boot until some Motorola-certified/signed/authorized software is loaded on the device again. They did not specify how that could be done, though. Still, even without the eFuse bricking, I find it outrageous that the Industry (including Motorola) expect their customers pay hundreds of dollars for a device that is then still owned by Motorola rather than that very customer. It's like selling something but still retaining ownership of it. Doesn't that make you feel strange, too?

Harald Welte: Implementing the TCAP protocol, heading towards OsmoSGSN SS7 support

Sun, 11 Jul 2010 02:00:00 +0000

The protocol by which traditional GSM core network components interact is called MAP (Mobile Application Part). MAP itself is a user of the TCAP (Transaction Capabilities Application Part) protocol, which in turn runs on a SS7 protocol stack (i.e. SCCP over MTP or M3UA or SUA over SCTP).

For those users of OpenBSC who have a need to interoperate with other GSM networks (roaming), the circuit-switched part of OpenBSC has so far relied on the use of a proprietary MSC (by means of the A interface). This closed MSC then talks MAP/TCAP/SS7 to roaming partners.

However, on the GPRS front, we now have OsmoSGSN. However, as opposed to the BSC on the circuit switched side, the SGSN directly interacts with the core GSM network components (both of the home network and the roaming partners).

So in order to run OsmoSGSN interacting with existing HLRs, we need to add a MAP/TCAP/SS7 interface to it. Once this has been done for the SGSN, we of course can do the same for the MSC-part that is currently integrated with OpenBSC.

As there are existing implementations of SCTP (inside the Linux kernel) and SUA (sualibrary), TCAP is the next step in the protocol stack that needs to be implemented. I've been digging into TCAP for the last week(s), and believe I finally understood every part of its operation.

You can think of TCAP as something that facilitates the transport of request-response type transactions over a datagram oriented transport layer. It intends to have lower overhead than a connection-oriented service (e.g. establishing TCP sessions) and supports features such as aggregating multiple user-messages (called components) in a single actual transport-layer message. The idea is to reduce the overhead of message headers and routing.

TCAP is (unfortunately) specified in ASN.1 and thus requires significant effort to parse and construct. Right now I'm using Lev Walkin's asn1c ASN.1 C code generator to generate the parser and constructor functions. The actual TCAP protocol logic is once again implemented in plain C, using the various concepts and utility functions established in OpenBSC (and now part of libosmocore).

The implementation is making good progress and I hope I can do some early testing in about a week from now, and successively move straight to the MAP protocol, implementing at least those parts that we need for GPRS authentication and attach / routing area updates.

Harald Welte: COSCUP 2010 conference schedule has been posted

Thu, 08 Jul 2010 02:00:00 +0000

The Schedule of the COSCUP 2010 conference has been posted on the conference homepage. I'm happy to see such a large number of talks from a wide range of speakers - including many friends from my time in Taiwan a couple of years back for Openmoko...

As it seems from this chinese blog entry, the organizers were overwhelmed by the number of attendee registrations, with all 610 available seats being occupied within 85 minutes of opening the registration. It seems they are in need of a bigger venue next year ;)

Rusty Russell: Superfreakonomics; Superplug for Intellectual Ventures.

Wed, 07 Jul 2010 10:52:14 +0000

I enjoyed Levitt & Dubner’s “Freakonomics”, and picked up the followup “Superfreakonomis” recently at an airport. The last chapter, however, was astonishing. The entire chapter was devoted to a glowing advertisement for Intellectual Ventures, pointing out that they own 20,000 patents “more than all but a few dozen companies in the world”, but of course “there is little hard evidence” that they are patent trolls.

But this bunch of wacky genius billionaires have solved global warming (much of which they dispute anyway) and can control malaria and prevent hurricanes from forming. Unlike the rest of the book which covers analysis of well-known facts and disputes them with insightful economic research, this chapter is so breathless and gushy that it makes me question the rest of the author’s work.

I first came across Intellectual Ventures when The Economist reversed their 100-year opposition to patents, and the only reason I could find was a similarly cheerleading piece about this company. (I had naively expected new research revealing some net positive of patents, or some such revelation).

Side note: when a respected information source covers something where you have on-the-ground experience, the result is often to make you wonder how much fecal matter you’ve swallowed in areas outside your own expertise.

So, what is IV actually doing? Buying up loads of patents and licensing them to companies who calculate it’s not worth the fight is patent trolling 101. Yet the scale they’re operating on puts them on new ground, and opens new opportunities. It seems obvious to get corporate investors on board by promising them immunity from patent claims. With enough patents you stop trying to license them one-by-one and just tax each industry at some non-negotiable rate. No doubt they have more tricks I haven’t even thought of, but these potential devices really do make them a new breed of Super Trolls.

Their efforts to actually attain their own patents could simply be more of the same, but it’s also a relatively cheap but clever PR exercise (as shown by their media treatment). This will help them when (legislative?) efforts are made to shut down patent trolls. I’m fairly confident that they’ll simply license rather than implement anything themselves; actually producing things requires much more work, and simply exposes you to others’ patents.

Without diving deeply into this, they seem to understand two things clearly:

They learnt from Microsoft that government-enforced monopolies are worth billions. Microsoft had copyright on software, this is patents.
Development is getting much cheaper, while patents are getting more valuable. Cheaper development is shown clearly by free software, open hardware and hackerspaces. Patent value increases as more of the world becomes a more profitable and enforceable patent target.

Now, I don’t really care if one company leeches off the others. But if they want to tax software, they have to attack free software otherwise people will switch to avoid their patent licensing costs. And if you don’t believe some useful pieces of free software could be effectively banned due to patent violations, you don’t think on the same scale as these guys.

Harald Welte: Family visit is keeping me busy

Mon, 05 Jul 2010 02:00:00 +0000

In case you're expecting a quick response from me these days, please apologize. I'm currently having family visiting me in Berlin, and I very much enjoy being the personal tourist guide for some days...

I shall be back to normal by the end of the week.

Harald Welte: Major update in OpenBSC GPRS/EDGE support

Fri, 02 Jul 2010 02:00:00 +0000

Through the last couple of days, I've been in extreme bug-squashing mode for the GPRS/EDGE code base in OpenBSC (mostly the OsmoSGSN program). I'm now at a point where I can reliably establish PDP contexts and access the Internet from a variety of different phones with different baseband chipsets and GPRS protocol stack implementations. All so-far-known bugs regarding fragmentation/reassembly, sequence numbering and other issues have been fixed. There definitely are plenty more, but we first need to find them.

Since it's working reliably now, it's quite fascinating what the various phones do after connecting to the GPRS network. Like Windows Mobile phones sending Netbios Name Service updates (and requests), which I think is funny considering that they are sent to a network that is typically considered to be the public Internet.

But to be fair and not anti-Windows, my Google/Android G1 also makes some https connections back to Google - and I don't know what they are for [yet].

In any case, with OpenBSC, OsmoSGSN and OpenGGSN anyone interested in doing true security (and privacy) research with mobile phones is now able to do so. Using those programs, you can run your own GPRS+EDGE network and can see first hand what your phones are doing on a cellular network, what kind of data they are sending back home. In this setup, there is no packet filtering, NAT, deep packet inspection and no intrusion detection systems between your PC and the IP stack on your phone.

Harald Welte: The reason why you see paging by IMSI in real-world GSM networks

Mon, 28 Jun 2010 02:00:00 +0000

During my work on airprobe and OsmocomBB I've been wondering why you see paging by IMSI in real-world GSM networks.

A quick recap: The IMSI is the world-wide unique serial number of your SIM. Since it is easy to identify and track people, the TMSI was introduced as a temporary identifier that is frequently re-allocated over encrypted channels. The only reason for the TMSI to exist is to prevent tracking of a subscriber by watching where his IMSI appears on the paging channel.

According to the theory, the IMSI is only used when first registering to any GSM network. At that time, a TMSI is allocated to the SIM card in the phone, and this TMSI is used for the next transaction(s). Later, this TMSI is re-allocated and re-allocated, but the IMSI shouldn't show up again in any paging requests.

Even if you switch mobile networks (i.e. in the roaming case), you would once send the IMSI as part of a LOCATION UPDATE REQUEST or IDENTITY RESPONSE, but the network has no need to page the SIM by IMSI.

So far the theory. If you look at the Paging Channel (PCH) of cells in real-world networks, you see a significant (10-20%) amount of paging requests that contain paging by IMSI. This seems strange on first sight, given the theory described above.

I have the following plausible explanation for this:

The VLR keeping the IMSI-TMSI mappings doesn't have non-volatile storage. This means at a VLR restart, all the TMSI allocations will be lost, and the network has to resort to paging by IMSI.
The VLR has a limited amount of RAM, which can store a limited number of IMSI-TMSI mappings. Especially if the operator is interested in saving money, the amount of memory is insufficient for all subscribers in the network. This means, the VLR will expire some old entries in the mapping table to store new entries. Thus, mobile phones whose last transaction with the GSM network was relatively long ago are likely candidates for such VLR expiration. Once a phone for an expired entry needs to be paged again, paging will happen by IMSI.
Last, but not least: GSM networks do not page a phone by the last known cell, but by the last known location area of the phone. A location area might be relatively big. This means that at any cell you will see a lot of paging messages, even for phones that are not even anywhere near this cell. If there is no response within the location area, the MSC might decide to do paging on a larger radius, possibly the entire MSC area. Since such MSC-wide paging is likely to occur for phones that haven't shown activity for a long time (and thus might have moved or disappeared without properly unregistering from the network), those are the exact same phones for which the IMSI-TMSI mappings have expired from the VLR. Thus, the rate of paging-by-IMSI looks disproportionately high.

So the relatively high percentage of paging by IMSI vs. TMSI should not be taken as a measurement with regard to the total number of transactions or even the total number of subscribers. It is simply the mechanics of the network resulting in a distortion of those figures caused by phones that have never properly unregistered from the network.

Harald Welte: Back from OpenBTS workshop

Sun, 27 Jun 2010 02:00:00 +0000

I've just returned back from the First OpenBTS workshop held by David Burgess and hosted by Dieter Spaar in south-east Bavaria (Germany). While I'm not involved with OpenBTS so far (except from using it occasionally), I still thought the community surrounding Free Software / Open Source in the GSM field is small enough to make me participate.

On the request of the participants, I also did a short demonstration of both OpenBSC and OsmocomBB. And just like I managed to crash OpenBTS by accidentally sending invalid messages, my OpenBSC demo crashed at some point [due to a not-yet-known bug regarding SMS delivery. I suppose the intrusive changes of the BSC/MSC split are to be blamed for that. But I don't mind, we need that split...

I definitely had a great time meeting the participants of the workshop. There definitely is a very diverse crowd with equally diverse reasons for their interest in using and/or deploying OpenBTS.

Finally, there was a chance to discuss the need for a common 'application interface' in both OpenBSC and OpenBTS. Using that interface, external applications (e.g. implementing USSD or RRLP) could be written in a way to work with both OpenBTS and OpenBSC. I hope we can get started on this soon and remove another bit of fragmentation in what is already a fairly small special interest community...

Given the excellent weather conditions, the motorbike ride to and from the venue went fine - despite being at 650 km distance from my home.

Harald Welte: Adding frequency hopping support to OpenBSC

Sun, 20 Jun 2010 02:00:00 +0000

During the last couple of days, I've been adding the bits required to support frequency-hopping BTSs in OpenBSC. Now everything looks great in the protocol traces - but unfortunately it still doesn't work, at least not with the Siemens BS-11 that I have access to.

Will continue to try to make it work. The big advantage of having a hopping BTS under our control is that we can define the hopping sequence - something quite useful once we get to the point where we'd like to add frequency hopping to the telephone-side stack (OsmocomBB).

The good news is that I had to fix lots of bugs in the A-bis OML dissector for wireshark that I wrote some time ago. It's now much more complete and definitely a big step further towards eventually getting it included in wireshark mainline.

Harald Welte: A fairy tale about ICCIDs, IMSIs and iPads

Tue, 15 Jun 2010 02:00:00 +0000

One of the big news of the last week is AT&T's leak of 114,000 iPad customer records including the e-mail address and ICCID

While that leak is certainly a big issue in itself, there are some people, most notably Chris Paget, who claim that this is much more serious than generally assumed. The main claim here seems to be that ...in order to translate an ICCID into an IMSI, you need to query the HLR.

I have been reading GSM protocol specifications on every level for the past years, and never have I seen the ICCID being mentioned anywhere. The GSM specifications do not require this information to be stored in the HLR, and the MAP protocol (used on the C interface between MSC and HLR, see 3GPP TS 29.002) does not even know how to encode/specify it.

Also, there is no technical need for it. The ICCID is never used nor needed in any part of the GSM protocol. Also, the GSM network typically doesn't store any information that is not absolutely necessary for its operation. The only identifier of a SIM card that the network protocols care about is the IMSI.

So unless the US operators in question have either some kind of proprietary extensions to both their HLR and the MAP protocol, there is to the best of my knowledge no way how you can relate the ICCID to the IMSI.

And thus, as a result, the IMSI-catcher attack described will not work since you don't know the IMSI of the SIM card (associated with the customer record) that you want to catch.

If anyone can show me hard technical facts about ICCIDs being used in the HLRs of the operators in question, I am happy to post here I was wrong. Otherwise, I would hope everyone else could also come down to the hard technical facts, i.e. which particular MAP message is used for this alleged ICCID-to-IMSI query.

UPDATE: As some people have discovered, the three US operators themselves have decided that they use the same number to generate both the ICCID and the IMSI. So if you have one, you can compute the other. No need for HLR access, no need for the MAP protocol. So the information leak is in fact unrelated to the GSM protocol but simply a matter of how unfortunate those particular three operators assign their unique identifiers.

Harald Welte: More thoughts on FSF action against Apple over GNU Go

Tue, 15 Jun 2010 02:00:00 +0000

Last week, I blogged about the FSF action against Apple. This week, I intend to add a bit to that.

As it has been pointed out to me, Apple has immediately removed the GPL-infringing software from its app store. This of course means they have refrained from further infringing the GPL. It is not publicly known if they have made a declaration to cease and desist or not.

So yes, by removing the software that was distributed in violation of the GPL terms, Apple has done legally the right thing: Reduce the danger/risk of committing further (knowing) infringement.

The FSF (and probably the Free Software community in general) of course want something else: For Apple to alter their app store terms in a way that would enable software authors to have Apple distribute their GPL licensed software in it. While this might be possible very easily with small modifications to their legal terms and to the implementation of the app store, it is probably not quite easy to make a legal claim and try to force this upon Apple.

Anyone always has the choice to either distribute GPL licensed software compliant with its license terms - or not distribute it at all. If Apple prefers the latter, this is very unfortunate (and you might call it anti-social or even anti-competitive) but something that they can very well do.

The only questions that I see remaining from a legal point of view: What about the previous GPL infringements? What can (and/or has) Apple to do in return to the previous distribution of infringing software? This is where the legal pressure of the copyright holders leaves room for negotiation. Instead of monetary damages (which don't really resolve what the GPL aims to do), there could possibly be a solution where Apple has to provide the GPL license text and complete corresponding source code to the Go program through their app store. And while they're at it, they might just solve the distributing source code for copyleft style licensed software problem in a generic way. Or they might just decide that they're stupid and stubborn and not interested in solving any problems in the first place.

Harald Welte: My take on the FSF action against Apple over GNU Go

Fri, 11 Jun 2010 02:00:00 +0000

About two weeks ago, the FSF announced that it has taken action against the Apple App Store over their distribution of GNU Go. This has apparently set off some people like lefty and triggered a length and wide debate.

I personally very much support the action the FSF has taken. Anyone involved in distribution of copyrighted material is required to do due diligence on checking that he actually has a license to do so. This is not really related to the GPL.

Yes, this means that I can take GPL enforcement action to a retail store that is selling/distributing infringing products, and I can make them provide a declaration to cease and desist from further infringements. Of course, that declaration would only be valid for this single retail store. This is why in our gpl-violations.org work, we always try to go after whatever entity is responsible for the majority or all of those infringements, rather than after a single store owner.

The reason for this is simple: In many cases, it is impossible for you as the rights holder to find out who sold the product to the retail store, and track the entire supply chain back to whoever caused the GPL violation in the first place. Also, some of those entities might reside in a different jurisdiction, so you go after the first element in the supply chain that is in your own jurisdiction, to minimize the legal risk for you as plaintiff and maximize the output in terms of your local market.

But the case with Apple is different. They are not a small retailer down the road, but the entity responsible for providing the infringing software to (almost?) all of its users. They are running that App store as a commercial company and earn money from running it (even if individual apps might be free of charge). Free Software and copyleft licenses like the GPL are a very real phenomenon in the software industry today, so they should better have thought about a proper solution, not just for GNU Go but for the tens of thousands of existing GPL licensed software projects which people might want to port or re-use in iPhone applications.

They are already doing all kinds of verification/checking/review of software for other reasons (things many people might call censoring), and as part of that process they could just as well determine the license of the software, and provide a source code download link from their store. What is the big deal? If they (or other similar app store / market / ... providers) had thought how to address the problem, there are easy and pragmatic solutions to solve them in the architecture of such a app store / marketplace system.

Also, the fact that the FSF is taking legal steps is not wrong. Even if some people might dispute whether they actually have a valid case or not (I believe they do): This is what legal cases are for: To create a clear legal situation for all participants in the dispute, and to set precedent for future similar cases. Even only from that point of view it is good that they're doing this case. At the end of it, the legal situation will be more clear, both for Apple as well as for people who want to distribute GPL licensed software through their store.

Rusty Russell: LWN Quotes Of The Week

Tue, 08 Jun 2010 05:01:38 +0000

I have been petitioning Jon Corbet to introduce a “Supporter” level subscription to LWN; I think given his failure to keep up with inflation and my need to experience conferences vicariously I feel deeply indebted to them.

That lead to me looking through LWN Quote of The Week; there’s no neat index of these things. And I’m not going to create one, but I did troll for my old quotes while procrastinating, and here’s my vanity list:

From the comments, it looks like I got the very first one. Wow. I’m sure there are missing ones (I was sure I got two weeks in a row once).

I’m behind quota for 2010 though; I was surprised my recent fight with Linus didn’t get in. I’ll try harder…

Harald Welte: Wanted: Packet traces of the MAP+TCAP based C/Gc interface

Sun, 06 Jun 2010 02:00:00 +0000

I'm looking for any example pcap files (packet traces) of the so-called "C" and "Gc" Interfaces, i.e. the interfaces of the HLR (Home Location Register). If anyone has such pcap files or can generate some, I would very much appreciate it.

The protocol levels I'm interested in is SCCP, TCAP and MAP. The lower layers (MTP) are not important now.

Specifically, I'm looking for traces of any of the following MAP operations:

updateLocation
cancelLocation
restoreData
sendParameters
updateGprsLocation
sendAuthenticationInfo
purgeMS
sendRoutingInfo
sendRoutingInfoForSM
reportSM-DeliveryStatus
readyForSM
noteSubscriberPresent
sendRoutingInfo
anytimeInterrogation
statusReport
{register,erase,activate,deactivate,interrogate}SS
sendIMSI
sendRoutingInfoForGprs
insertSubscriberData
deleteSubscriberData
checkIMEI

If anyone is able to produce the respective traces, I would appreciate it a lot. I'd need them as examples to make sure I fully understand the TS 09.02 in combination with Q.77x before actually starting to implement it...

Harald Welte: First functional HTTP transfer in my own GPRS/EDGE network

Thu, 03 Jun 2010 02:00:00 +0000

Today marks the day where finally, after months of (non-full-time) work, I have made the first successful HTTP connection through my own GPRS/EDGE network.

Ever since we started to seriously get into OpenBSC to run GSM networks, I've been looking forward to running GPRS networks, too. What most people don't know: GPRS is radically different from GSM. It basically only shares the frequencies and timeslot architecture of the physical layer, while having it's own layer1, layer2 and various other protocol layers. Also, its signalling and data completely bypass the usual BSC and MSC components of a GSM core network.

So what I've been working on is now called OsmoSGSN. Using OpenBSC, you can provision an ip.access nanoBTS (or any other BTS with a Gb Interface) to broadcast the GPRS/EDGE capabilities to the handsets. The BTS then establishes the Gb interface (consisting of NS and BSSGP) to the SGSN.

The SGSN takes care of GPRS Mobility Management (GMM) and Session Management(SM) in the signalling plane, as well as the LLC + SNDCP protocol layers. On the other end, it uses the GTP protocol to connect to a GGSN. In our case, this is the OpenGGSN project which I recently adopted.

OpenGGSN then creates a virtual network device (tun0), through which the actual IP packets are entering/leaving the GPRS network. From there you can route and/or NAT them just like any other IP packets.

The current code is still incomplete in many places and known to be unstable. But it's really rewarding that after a long time of development, layer after layer of the stack, finally actual TCP/IP can be provisioned to phones.

The code is in the current master of the openbsc git repository, but I don't think there's much point in trying it just yet. I suppose in a week from now things should be much more stable.

Harald Welte: The Linux-Kongress 2010 CfP is about to close

Mon, 31 May 2010 02:00:00 +0000

The Linux-Kongress 2010 Call for Proposals is about to close.

So if you have anything interesting related to Linux that you would like to talk about at the 2010 incarnation of one of the most traditional Linux conferences, this is your last chance. There is no excuse, do it right now!

Harald Welte: UPS sends me an invoice over 1 Euro-cent

Fri, 28 May 2010 02:00:00 +0000

Yesterday I received this letter from the local UPS subsidiary in Germany.

This is nothing uncommon, as I often import some electronics parts or other equipment from outside the EU, on which I need to pay customs duties and/or import VAT. In such cases, they typically collect an estimated amount as COD (cash on delivery) and then send an invoice about the difference (if any).

The funny part in this case now is: The grand total after subtracting my COD payment is EUR 0.01 - in words: One Euro-cent. They really want me to do a bank transfoer or write them a cheque over 1 cent !?!

One wonders to what grand total the expenses for the paper, printing, postage, banking transfer fees and accounting fees on the UPS side will amount to for processing something like this.

I wonder what would happen if I didn't pay that 1 cent. Would they actually try to sue me? Probably simply stop delivering packets to me, which I cannot afford and thus rather pay that single cent...

Harald Welte: OpenGGSN Version 0.90 released

Tue, 25 May 2010 02:00:00 +0000

Only three weeks ago I blogged about OpenGGSN, a seemingly-abandoned Free Software implementation of the GGSN node of the GPRS core network.

Things have developed quite a bit ever since. As the original author didn't respond to any of my mails and sourceforge.net was not able to reach him either, they have approved my the 'abandoned project takeover' (APT) request.

I've now switched the project from CVS to git, removed links to the non-existing openggsn.org homepage and released version 0.90, containing nothing less than a fix for remote DoS vulnerability that was pending for more than 5 years.

So far I'm only exercising the PDP context activation/deactivation parts of OpenGGSN from OsmoSGSN (the SGSN I write as sister-project to OpenBSC), but I hope we can use OpenGGSN in a production GPRS network soon...

Rusty Russell: Typesafe callbacks in C (and gcc)

Mon, 24 May 2010 02:28:28 +0000

A classic pattern in C is to hand a generic callback function around which takes a “void *priv” pointer so the function can take arbitrary state (side note: a classic anti-pattern is not to do this, resulting in qsort being reimplemented in Samba so one can be provided!).

The problem with this pattern is that it breaks type safety completely, such as in the following example:
int register_callback(void (*callback)(void *priv), void *priv); static void my_callback(void *_obj) { struct obj *obj = _obj; ... } ... register_callback(my_callback, &my_obj);

If I change the type of my_obj, there’s no compiler warning that I’m now handing my callback something it doesn’t expect.

Some time ago, after such a change bit me, I proposed a patch to lkml to rectify this, using a typesafe callback mechanism. It was a significant change, and the kernel tends to be lukewarm on safety issues so it went nowhere. But these thoughts did evolve into CCAN’s typesafe_cb module.

The tricksiness…

More recently, I tried to use it in libctdb (the new async ctdb library I’ve been toying with), and discovered a fatal flaw. To understand the problem, you have to dive into how I implemented typesafe_cb. At its base is a conditional cast macro: cast_if_type(desttype, expr, oktype). If expr is of type “oktype”, cast it to “desttype”. On compilers which don’t support the fancy gcc builtins needed to do this, this just becomes an unconditional cast “(desttype)(expr)”. This allows us to do the following:
#define register_callback(func, priv) \ _register_callback(cast_if_type(void (*)(void *), (func), void (*)(typeof(priv)))

This says that we cast the func to the generic function type only if it exactly matches the private argument. The real typesafe_cb macro is more complex than this because it needs to ensure that priv is a pointer, but you get the idea.

Now, one great trick is that the callback function can take a “const” (or volatile) pointer of the priv type, and we let that work as well: we have a “cast_if_any” which extends “cast_if_type” to any of three types:
#define typesafe_cb(rtype, fn, arg) \ cast_if_any(rtype (*)(void *), (fn), \ rtype (*)(typeof(*arg)*), \ rtype (*)(const typeof(*arg)*), \ rtype (*)(volatile typeof(*arg)*))

The flaw…

If your private arg is an undefined type, typeof (*arg) won’t work, and you need this to declare a const pointer to the same type. I have just filed a bug report, but meanwhile, I need a solution.

The workarounds…

Rather than use cast_if_any, you can insert an explicit call to the callback to evoke a warning if the private arg doesn’t match, then just cast the callback function. This is in fact what I now do, with an additional test that the return type of the function exactly matches the expected return type. cast_if_type() now takes an extra argument, which is the type to test:

#define typesafe_cb(rtype, fn, arg) \ cast_if_type(rtype (*)(void *), (fn), (fn)(arg), rtype)

cast_if_type does a typeof() on (fn)(arg), which will cause a warning if the arg doesn’t match the function, and the cast_if_type will only cast (fn) if the return type matches rtype. You can’t test the return type using a normal test (eg. “rtype _test; sizeof(test = fn(arg));”) because implicit integer promotion makes this compile without a warning even if fn() returns a different integer type.

Unfortunately, the more general typesafe_cb_preargs() and typesafe_cb_postargs() macros lose out. These are like typesafe_cb but for callbacks which take extra arguments (the more common case).

/* This doesn't work: arg might be ptr to undefined struct. */ #define typesafe_cb_preargs(rtype, fn, arg, ...) \ cast_if_any(rtype (*)(__VA_ARGS__, void *), (fn), \ rtype (*)(__VA_ARGS__, typeof(arg)), \ rtype (*)(__VA_ARGS__, const typeof(*arg) *), \ rtype (*)(__VA_ARGS__, volatile typeof(*arg) *))

We can’t rely on testing an indirect call: we’d need example parameters to pass, and because they’d be promoted. The direct call might work fine but an indirect call via a different function signature fail spectacularly. We’re supposed to increase type safety, not reduce it!

We could force the caller to specify the type of the priv arg, eg. “register_callback(func, struct foo *, priv)”. But this strikes me as placing the burden in the wrong place, for an issue I hope will be resolved soonish. So for the moment, you can’t use const or volatile on callback functions:

/* This doesn't work: arg might be ptr to undefined struct. */ #define typesafe_cb_preargs(rtype, fn, arg, ...) \ cast_if_type(rtype (*)(__VA_ARGS__, void *), (fn), (fn), \ rtype (*)(__VA_ARGS__, typeof(arg)))

Harald Welte: dfu-util release 0.1 has been announced

Mon, 24 May 2010 02:00:00 +0000

Back in the early days of my work at Openmoko, I had come up with the idea to use the standardized USB Device Firmware Upgrade (DFU) protocol for flashing firmware to the Neo1973 and later FreeRunner phones. This encompassed a DFU device implementation that is part of the Openmoko u-boot variant (and has meanwhile been merged in one of the u-boot successor projects) as well as a tool for the host PC called dfu-util.

Since DFU is meant to be device and vendor-agnostic, I tried to code closely to the spec. This meant that it in fact was compatible to other devices, and some folks e.g. used it to flash firmware into their USB-Bluetooth controllers from CSR.

However, there never was any official information how to use dfu-util outside the context of Openmoko, and even more specifically: There never were any official releases.

Stefan Schmidt has stepped up to change this and maintain dfu-util as well as make official releases. The first such release has now been made at the new dfu-util project homepage.

Harald Welte: I'll be presenting at COSCUP 2010 in Taiwan

Thu, 20 May 2010 02:00:00 +0000

I have just received the great news that my attendance of the COSCUP 2010 conference in Taiwan is now confirmed. Thanks to COSCUP for inviting me!

I'll be participating in the legal track and presenting on GPL license compliance. The exact title and abstract is not yet decided.

As usual, I'm really looking forward at any chance to visit Taiwan, and the trip this August is definitely no exception. Now I only need to decide how long I'm going to stay before/after the conference...

Harald Welte: Heading off to Europe's largest Goth festival

Thu, 20 May 2010 02:00:00 +0000

Despite lots of very exciting work at this time, and a distinct lack for progress on my various 'just for fun' software/hacking projects, I'll be visiting Wave-Gotik-Treffen from tomorrow on. This means that I'll be listening to some fine music and will hopefully have a most enjoyable time offline.

Don't expect me to read or answer e-mails or get any work (paid or unpaid) until at some point Tuesday next week.

Harald Welte: Doing even more encapsulations than the GPRS Gb interface already has

Wed, 19 May 2010 02:00:00 +0000

Back in October 2009, I blogged about the incredibly deep protocol stack on the GPRS Gb interface between a BSS and the SGSN.

Today I had the pleasure of implementing an even more odd variant of the Gb interface, where NS does not get encapsulated in UDP/IP/Ethernet, but in FrameRelay/GRE/IP/Ethernet. The total protocol stack thus then looks like: HTTP/TCP/IP/SNDCP/LLC/BSSGP/NS/FR/GRE/IP/Ethernet, with an optional PPP between IP and SNDCP. If anyone does the math to calculate the total protocol overhead, please let me know[...

The reason for that oddity is apparently that there are Cisco and other routers that can encapsulate Frame Relay in GRE. So using a old circuit-switched SGSN with E1 interfaces and such a router, you can convert from Frame Relay on E1 to Frame Relay on GRE/IP/Ethernet.

Both the Gb Proxy as well as the upcoming OsmoSGSN use the same NS implementation, i.e. they can now both talk NS/FR/GRE and the NS/UDP variants - even at the same time, as the encapsulation is a property of each individual NS-VC.

Harald Welte: Back from a week of GSM/GPRS protocol coding/testing in Iceland

Sun, 16 May 2010 02:00:00 +0000

With only 16 hours delay (which isn't all that much considering the volcanic ash situation) I arrived back in Berlin from one week of OpenBSC software hacking, particularly on the GPRS side of things.

It was really nice to see to what extent OpenBSC software is already used at On-Waves, providing GSM and now also GPRS services to thousands of users.

My work was mostly focused on the Gb-Proxy, a multiplexer/proxy for GPRS Gb links running the NSIP (NS-over-IP) protocol. It combines elements of the idea of a network address translator with that of a proxy, combined with a little bit of packet-based routing. This really makes me feel like I'm back to packet-switched networking, which is great. Especially the fact that we use the VTY code from quagga and its interactive command line sometimes lets you forget that you're not working with classic TCP/IP routing daemons or the like ;)

Aside from that, I continued my work on the upcoming OsmoSGSN, using which we will be able to run an autonomous GPRS network with no dependency on external proprietary components. In this setup, the PCU in the BTS connects over Gb/IP to OsmoSGSN, which then talks over GTPv1 to the OpenGGSN.

Also, work was spent on an abstract rate_counter implementation (now part of libosmocore). The idea is to have a counter that will count certain events (like number of packets/bytes, number of link failures, etc), but also keep a small history about how many of those events happened in the last second, last minute, last hour and last day. There is also common code to store those counters in the database, as well as to print them on the VTY. The new counters are so far only used in the GB-Proxy, but they will soon likely be added to OpenBSC (bsc_hack) and other programs of our Free Software GSM network portfolio.

Harald Welte: Heading for 4 days of Iceland to work on OpenBSC GPRS

Mon, 10 May 2010 02:00:00 +0000

Having just returned from Croatia the day before yesterday, I'm about to head on a four-day trip to Iceland, where I'll be doing some testing and bug fixing of the current OpenBSC GPRS support at On-Waves.

Zecke is also going to be there, working on other aspects of OpenBSC. This will make the trip even more fun!

Harald Welte: CECT C3100: Not a phone, but a flashlight with integrated phone

Sun, 09 May 2010 02:00:00 +0000

I've seen many [mobile] phones in my life, but nothing like the CECT C3100 so far. It's made of the cheapest hard plastic, like cheap kids toys. In addition to the phone keyboard, it has a mechanical switch on its side. If you slide that switch, five powerful bright white LEDs at the top of the phone will turn the entire device into a flashlight.

However, it is one of the most basic phones with one of the older/simpler MTK baseband chips inside (MT6223). Also, as we have determined by a PCB delamination analysis, the test pads next to the MT6223 really are its ARM JTAG pins.

JTAG is something not commonly found in MTK phone designs, but it is definitely a big win for bootstrapping any system-level software such as drivers on the unit.

Right now I don't have the time to work on MT6223, we still have many issues to fix in the current Ti Calypso code. But I can't wait to find time to see if we can extend our hardware support to MTK GSM chipsets...

Harald Welte: GPRS progress in OpenBSC

Tue, 04 May 2010 02:00:00 +0000

In recent weeks, I have been able to pick up my work at GPRS support for OpenBSC again. What has been done is:

Add OML support to configure nanoBTS for EDGE
Add RR (System Information) support to indicate EDGE support
Make a OpenBSC + nanoBTS setup inter-operate with an existing SGSN
Develop a proxy that can aggregate the Gb-interfaces of multiple BTS into one Gb link to a real SGSN. This way the SGSN has only one Gb link for all the cells under the control of a BSC, as opposed to one Gb link for each and every BTS

What I'm working on now is the actual SGSN implementation. The SGSN is mainly responsible for GPRS mobility management (GMM) and for terminating the Layer2 (LLC) protocol from the MS. This is very different from circuit-switched GSM, where Layer2 (LAPDm) already terminates at the BTS.

The layering stack of GPRS is a real nightmare, I am sure I have indicated this in this blog before. The Current OsmoSGSN code (available from the regular openbsc.git repository) implements the NS, BSSGP and LLC layers, as well as the basic GSM 04.08 GPRS Mobility Management messages like GPRS ATTACH/DETACH and ROUTEING AREA UPDATE. The LLC code is still somewhat limited, but for the time being it is sufficient.

What drove me crazy for a couple of days is the number of parameters that are exchanged at PDP CONTEXT ATTACH time. There are no less than 26 different quality of service (QoS) parameters negotiated (see struct gsm48_qos at the bottom of this link), each of them from a wide range of values. It's almost impossible to imagine more than 1% of all the possible combinations have ever been used in production networks. The QoS parameter negotiation works by the phone sending a list of requested parameters, to which the SGSN responds with its selected parameters. My first thought was: Lets be smart and simply echo back the QoS parameters - the phone must accept what it has requested. That didn't work either: While the QoS structure is the same in both ways, the actual values in uplink and downlink directions are encoded differently. Who on earth defines such an encoding?

Next item was the XID exchange which is at the boundary between LLC and the SNDCP (Sub-Network Dependent Convergence Protocol). It works like this: The phone proposes an endless list of parameters, which the SGSN can evaluate, and then depending on the parameter type either negotiate up or down. According to the spec, sending an empty XID response should mean "I agree with all your parameters". However, at least those phones that I tested were not happy with that. So I decided to simply send back the entire XID block to the phone. And believe it or not, as opposed to the QoS parameters, this time it even worked

So now I'm facing the implementation of the actual SNDCP-to-GTP interworking, which is nothing less but the guts of the SGSN. GTP is the protocol used on the GGSN side. At least this time, GTP is sent directly over TCP or UDP, i.e. the stacking inside the SGSN is only one layer deep, while on the Gb-interface it is four (NS,BSSGP,LLC,SNDCP).

SNDCP interacts with the GPRS Mobility Management, GPRS Session Management (both GSM 04.08 over LLC), the GTP interface to the GGSN, as well as other parts. I expect many pitfalls on the way to getting it working, and given the complexity involved I have already decided to stick much closer to the specification than I usually did with the OpenBSC work. This means properly implementing all the state machines with all their transitions, exceptions and timers. I'm sure it's going to be "fun". The good part of it is: Most of the SGSN will be re-used once we finally get around adding support for 3G/UMTS/WCDMA cells.

Harald Welte: OpenGGSN - An open source GGSN implementation

Tue, 04 May 2010 02:00:00 +0000

As a friend pointed out to me at exactly the right point in time: there already is an Free implementation of a GGSN. In case you don't know what a GGSN is: It is one of the two core components of a GPRS network. So, in order to extend a OpenBSC GSM network with GPRS support, there are two components required: The SGSN (on which I'm working currently, project name OsmoSGSN), and the GGSN. Due to the good news about OpenGGSN, I'm quite confident that I will not need to implement the latter part.

OpenGGSN is not only a Free Software implementation of the GGSN, but it is also licensed under GPLv2, making it compatible with the OpenBSC codebase (which is GPLv2 or later). This means I will be able to link the OpenGGSN-provided libgtp library (implementing both sides of the GTP protocol between SGSN and GGSN) from OsmoSGSN, further reducing the amount of work required to get this working.

However, despite seeming like a fairly advanced/complete implementation of the GGSN specification: OpenGGSN seems like a project that was abandoned many years ago. The latest CVS commit is from 2005, and all of the bug fixes that people have submitted to the bug tracker have not been merged. The homepage is defunct, and the openggsn.org domain name seems to have been expired (and grabbed).

I've tried to contact the author by e-mail about his intentions for the project, let's see if there is any response. Meanwhile, I have generated a git repository from the OpenGGSN CVS repository at sourceforge and applied all the pending fixes to a local branch. See my related mailing list post for more information.

Harald Welte: Security product technical details need to be disclosed while importing to China

Sun, 02 May 2010 02:00:00 +0000

According to this report at The Register, there are some new government regulations about the import of certain security products into China, including Smartcards, firewalls and routers. While importing the goods, the importer needs to submit the technical details to a government panel in order to get the import license.

However, the article claims there are no further details on what exactly needs to be disclosed. Anyone who knows more details: I'd be more than interesting to hear about them - maybe there's even an English translation of the respective law or regulation?

I think it is a most reasonable policy that a country can adopt. Security products whose operation relies on its secrecy are useless anyway. The concept of security-by-obscurity has never worked and has been proven wrong many times, e.g. in the NXP Mifare Classic, DECT cipher/authentication, GSM A5 cipher and many other proprietary encryption schemes.

The only thing the Chinese regulators are doing wrong: According to their rules, the information must be disclosed to a closed government panel. Instead, they should require such information to be published publicly, or at least to be released in full detail to all customers of the respective product.

Harald Welte: Attending DORS/CLUC 2010 in Zagreb next week

Fri, 30 Apr 2010 02:00:00 +0000

I'm looking forward to attend DORS/CLUC 2010 in Zagreb/Croatia next week. DORS/CLUC is a small but nice event, with a group of very warm and welcoming organizers. I've been there a couple of times before and always had a very good time.

Harald Welte: Linux-Kongress 2010: Call for Proposals closes soon

Fri, 30 Apr 2010 02:00:00 +0000

This years will mark the 17th incarnation of Linux Kongress. It is scheduled from September 21st through 24th in the city of Nürnberg (aka Nuremberg), which (as a personal side note) also happens to be the city where I was born and where I've grown up.

The Call for Proposals is out for quite some time, and will last for another month until June 1st. So if you have something exciting to talk about that is related to Linux and of technical nature: Please submit your proposal soon. Looking forward to listening to your presentation at LK2010 :)

Harald Welte: I'll be presenting at the SSTIC 2010 conference

Thu, 29 Apr 2010 02:00:00 +0000

I've been invited (as apparently the only non-french-speaker) to present at the SSTIC 2010 conference in Rennes/France.

There will be two presentations: One about OpenBSC, the other about OsmocomBB. Both will cover the use of the respective projects in the context of doing security analysis on a GSM protocol level.

Harald Welte: Sony faces class action lawsuit on removing the Linux support on PS3

Thu, 29 Apr 2010 02:00:00 +0000

As reported, a class action lawsuit has been filed against Sony in the US for removing the so-called Other OS feature from Playstation 3. The PS3 was originally advertised as being able to run Linux, and I know a number of people who have bought it for exactly that reason. Removing that feature after the purchase is thus significantly reducing the value of the product to many of its users.

I can only hope that this lawsuit will be successful. After I have bought a product, I own it and I decide what to do with it, not the original manufacturer. There have been somewhat related cases where Amazon removed already purchased books from the eBook readers of their customers. This is simply insane. With the ever growing power that corporations try to achieve over what their customers do or don't do, the outcome of this case might have significant importance for consumer rights in the decades to come.

Harald Welte: The mid-term future of WebOS seems safe

Thu, 29 Apr 2010 02:00:00 +0000

After HP announced its acquisition of Palm, I think we can be sure that the mid-time future of WebOS seems quite safe. I also expect mechanically much better hardware among the devices they will ship.

However, the acquisition could also mean a shift in politics, i.e. cause the new devices to be locked down with cryptographically signed kernel images. One of the big advantages of the existing Pre and Pixi is that they are not locked down and that as a user you can take full control over the device.

Another policy that might come under re-evaluation is the relationship between the WebOS Application Market and the third-party application installers like Preware.

Lets hope the managers responsible for WebOS future realize that their chance is to be less restrictive and more open than most of the competition - including most Android devices. At least, one could hope, HP has quite some experience with Linux and the Open Source community in other areas of their business.

Harald Welte: Chaosradio Express 151: ARM CPU Architecture (German)

Wed, 28 Apr 2010 02:00:00 +0000

I'm a bit late with this: The Chaosradio Express #151 podcast on the ARM CPU architecture has been released a week ago. I had a most pleasant experience spending about 90 minutes getting interviewed by Tim Pritlove.

I'm sorry for all the non-German-speakers. But Chaosradio Express is a German medium, made by and for German hackers :)

Harald Welte: Working on GPRS support for OpenBSC again

Tue, 27 Apr 2010 02:00:00 +0000

This has been on my TODO list for at least the last six months or so: Growing the experimental GPRS branch of OpenBSC into something more useful.

Right now, you can use OpenBSC with a GPRS-capable BTS - but only if you have an existing SGSN to serve the Gb interface of the BTS. This somehow defeats the point. We want to offer a 'GSM network in a box' solution, where no other non-free Software is required to run a fully functional small network.

So right now I'm cleaning up the 08.16 (Network Services) Implementation, and will move my way up through the existing 08.18 (BSSGP) and LLC code that I wrote some time ago.

With some luck, in a couple of weeks we should be able to run a self-sufficient combined GSM + GPRS (+ EDGE) network out of OpenBSC.

David Miller: Couple of strcmp tricks...

Sun, 25 Apr 2010 04:43:00 +0000

Like for strlen it's pretty easy to make a loop which checks a long word at a time. The less easy part is making the code run cheaply in constant time when we exit that loop.

We're going to work with two constants:

#define	STRCMP_CONST1	0x7f7f7f7f7f7f7f7f
#define	STRCMP_CONST2	-0x0101010101010101

We have two calculations, one for the inner loop and one for the post-loop calculations. Two are necessary because we want to minimize the cycle count in the loop and we only care there if there exists a zero byte somewhere. Whereas in the post-loop exit code we have to know precisely which byte the zero resides in.

The inner loop runs roughly like (in pseudo C):

	while (1) {
		s1 = *s1_word++;
		s2 = *s2_word++;
		g2 = s1 + STRCMP_CONST2;
		g1 = s1 | STRCMP_CONST1;
		s_xor = s1 ^ s2;
		if (s_xor)
			break;
		if (g2 & ~g1)
			return 0;		
	}

We specifically prioritize the inequality test before there "is there a zero byte in s1" test. And the inner loop "zero byte present in 's1'?" test is:

	(s1 + -0x0101010101010101) & ~(s1 | 0x7f7f7f7f7f7f7f7f)

It's one of several ways to test for a zero byte in a word in 3 instructions. But as mentioned it's imprecise in that due to cascading overflows from the addition, the zero marker left in the mask result might not be in the actual zero byte. That's important in the post-loop exit code calculations so we'll use something else, which is:

	tmp1 = ~(s1 | 0x7f7f7f7f7f7f7f7f);
	tmp2 = ~((srcword1 & 0x7f7f7f7f7f7f7f7f) + 0x7f7f7f7f7f7f7f7f);
	x = (tmp1 & tmp2);

"x" will have a "0x80" value in every byte that was zero in "s1" and zeros elsewhere. In the inner loop we already calculated "(s1 | 0x7f7f7f7f7f7f7f7f)" so we can reuse it and simply negate it.

How does this clever calulation work it's magic? 'tmp1' records every byte of the word that has bit 7 clear, and gives us a "0x80" in such bytes and a zero in all others. 'tmp2' records all the bytes that have all bits below 7 (0x7f) clear, leaving a 0x80 in all such bytes and zeros elsewhere. So "tmp1 & tmp2" is "all bytes that have all bits clear". Get it?

Now, using "x" we can see which comes first, a byte miscompare or a zero. Note we already have this "s_xor" thing calculated in the inner loop, and we'll use that here.

	ret = ((unsigned)s1 = (unsigned)s2) ? -1 : 1;
	low_bit = x >> 7;
	if (low_bit > s_xor)
		ret = 0;

And we're done.

"low_bit" is the "x" value shifted down by 7 bits so that we have a "0x01" in every byte that was zero in "s1". With big-endian byte ordering, if we simply compare this "low_bit" with the "s_xor" a larger value of "low_bit" indicates that the zero byte comes first. And in such a case the strings we scanned were equal and we should return "0".

A lot of strcmp implementations have performance which is dependent upon which byte in the long word miscompares (they simply scan a byte at a time when they exit the loop) or their location agnostic code is much more expensive than it needs to be.

Here's an icebag for your brain, I can see the steam coming out of your ears after reading all this.