Letzte Woche spielte Deutschland auf Zypern. Nach 90 Minuten stand es 1:1 - zum ersten Mal seit der WM hatten wir nicht gewonnen und natürlich war das ein gefundenes Fressen für Presse, Funk und Fernsehen. Da konnte man doch endlich mal diese überbezahlten Idioten abkanzeln, die einen Haufen Kohle dafür bekommen, gegen den Ball zu treten und dann doch nur einen Punkt bei einem Niemandsgegner wie Zypern zustande bekommen. Eine Freundin von uns hat sogar alle 10 Minuten den Live-Ticker gecheckt, ob wir auch ja noch zurück liegen.
Dieses Ereignis hat mir mal wieder zu Denken gegeben. Im letzten Sommer waren wir noch alle stolz auf unser Team - der Fußball war gut und die Euphorie wurde nicht einmal durch die Niederlage im Halbfinale gebremst. Mit Eintreten der kalten Jahreszeit scheint sich nun aber auch wieder des Deutschen Gemüt zu kühlen.
Und was finden wir nun, tief in den Wurzeln unserer politisch korrekten und historisch bereinigten Leitkultur? Wir sehen, dass es da Leute gibt, denen es besser geht als uns - Fussballer, die wenig leisten und trotzdem unglaublich viel mehr Geld, Ruhm und Status erlangen als wir das je in unserem Leben erreichen werden. Für viele ist das offenbar ein Grund, wieder alles schlecht zu machen. Willkommen zurück in der Deutschen Neidkultur.
Hat sich einer der Neider eigentlich schon mal überlegt, warum diese Jungs so reich sind? Ganz einfach: weil es Leute gibt, die bereit sind, für Fussball zu bezahlen. Fans zahlen, weil sie Fußball lieben. Firmen zahlen, weil sie beim Fußball Werbung machen und die zahlenden Fans dann hinterher in den extra länger geöffneten Supermarkt rennen, um all die tollen Produkte zu erwerben. Dieses Spiel ist ein relevanter Teil unserer Marktwirtschaft.
Sportler sind ein wichtiger Teil dieses Spiels - warum sollen sie also nicht auch finanziell daran teilhaben? Können wir uns nicht einfach an ihnen erfreuen und ihr Können bewundern?
Last.FM: Recently listened
Thursday, November 23, 2006
Wednesday, November 01, 2006
Exokernel snow
It's been a while since I posted some stuff on this blog. I really didn't know what to talk about at the moment. Don't want to go into details about football - Dynamo is a shame these days and I can only hope that this winter is not going to be as frustrating as last year's. There's also not much to talk about work - I am currently participating in porting our Device Driver Environment to support Linux 2.6 drivers. This is quite interesting for me, but there is not yet much to talk about.
The first rainy flakes of snow are cumulating on the rooftop window above me and I realize that although nights are getting longer, days becoming colder and the winds getting stronger, there are still nearly two months left until Christmas. (Best wishes to my friends Lars, Rene and Dina currently being in "icy" Russia... *wave*).
Winter term has started and we are doing some teaching again. Besides the normal exercise stuff and some advanced lectures on microkernel design, we are once again trying to start a Paper Reading Group. Neal, one of our students, is kind of interested in scientific papers from the old days. He proposed reading a lot of papers from the 1960's and 1970's. Today was the first meeting and as I'm always keen to gain more experience in giving talks, I volunteered for one of the first presentations. My talk was based upon two papers on Exokernels by Frans Kashoek, Dawson Engler and others:
The Exokernel does not do any resource management apart from tracking who is in possession of a resource (allocation, revocation, sharing). Security policies are implemented in the applications with only minimal support being necessary from the exokernel.
On top of the low-level hardware abstractions, exokernels implement higher-level operating system services in a library. These so-called libOSes are linked to each application and let application performance be at least as performant as traditional operating systems. The guys even implemented some custom-tailored applications that exploit the low-level exokernel layer in ways that enable very high performance.
I think I learned some interesting ideas, even though I don't agree with all the statements from these papers. Moreover, my conclusion is, that I need to read more scientific papers. Just sitting around all day, waiting for ideas to become rich, will not suit me well. Next is a paper about network packet filtering dating back to 1986...
The first rainy flakes of snow are cumulating on the rooftop window above me and I realize that although nights are getting longer, days becoming colder and the winds getting stronger, there are still nearly two months left until Christmas. (Best wishes to my friends Lars, Rene and Dina currently being in "icy" Russia... *wave*).
Winter term has started and we are doing some teaching again. Besides the normal exercise stuff and some advanced lectures on microkernel design, we are once again trying to start a Paper Reading Group. Neal, one of our students, is kind of interested in scientific papers from the old days. He proposed reading a lot of papers from the 1960's and 1970's. Today was the first meeting and as I'm always keen to gain more experience in giving talks, I volunteered for one of the first presentations. My talk was based upon two papers on Exokernels by Frans Kashoek, Dawson Engler and others:
- Exokernel: An Operating System Architecture for Application-Level Resource Management ( Dawson R. Engler, M. Frans Kaashoek, James O'Toole Jr.)
- Application Performance and Flexibility on Exokernel Systems ( M. Frans Kaashoek, Dawson R. Engler, Gregory R. Ganger, Héctor M. Briceño, Russell Hunt, et al.)
The Exokernel does not do any resource management apart from tracking who is in possession of a resource (allocation, revocation, sharing). Security policies are implemented in the applications with only minimal support being necessary from the exokernel.
On top of the low-level hardware abstractions, exokernels implement higher-level operating system services in a library. These so-called libOSes are linked to each application and let application performance be at least as performant as traditional operating systems. The guys even implemented some custom-tailored applications that exploit the low-level exokernel layer in ways that enable very high performance.
I think I learned some interesting ideas, even though I don't agree with all the statements from these papers. Moreover, my conclusion is, that I need to read more scientific papers. Just sitting around all day, waiting for ideas to become rich, will not suit me well. Next is a paper about network packet filtering dating back to 1986...
Wednesday, July 26, 2006
Summer School on Dependable Software Systems
After finishing my diploma and starting to work as a PhD student at the TU Dresden Operating Systems Group in the beginning of July, I attended the Summer School on Dependable Software Systems last week. It took place in Lugano, Switzerland. Six researchers from USI Lugano, Carnegie-Mellon University and the University of Milano presented the best of their knowledge and research.
Natasha Sharygina from USI talked about using model-checking for analysis of large C applications. She gave an overview of the SATABS tool she is involved in and gave some thoughts about veryfing mobile applications.
Fernando Perdone from USI introduced reliable broadcast mechanisms and the problems related to this area.
Peter Lee from CMU introduced Proof-Carrying Code as a method to proof the correctness of an applicatoin to a customer.
Mary Shaw from CMU talked about Ultra-Large-Scale Systems and about everyday dependability where we have to consider changing users needs during workdays although the effect of failures might not be critical like in an airplane, but only inconvenient to a user.
David Garlan talked about formalizing software architectures, so that even high-level architectures may be checked for consistency and conformance with respect to certain rules posed on a system.
Carlo Ghezzi talked about changes in software development, targetting mobile and dynamic systems instead of static and monolithic ones.
Natasha Sharygina from USI talked about using model-checking for analysis of large C applications. She gave an overview of the SATABS tool she is involved in and gave some thoughts about veryfing mobile applications.
Fernando Perdone from USI introduced reliable broadcast mechanisms and the problems related to this area.
Peter Lee from CMU introduced Proof-Carrying Code as a method to proof the correctness of an applicatoin to a customer.
Mary Shaw from CMU talked about Ultra-Large-Scale Systems and about everyday dependability where we have to consider changing users needs during workdays although the effect of failures might not be critical like in an airplane, but only inconvenient to a user.
David Garlan talked about formalizing software architectures, so that even high-level architectures may be checked for consistency and conformance with respect to certain rules posed on a system.
Carlo Ghezzi talked about changes in software development, targetting mobile and dynamic systems instead of static and monolithic ones.
Sunday, May 21, 2006
[L4] Speeding up L4Linux task creation
It's L4Linux once again. As mentioned the last time, L4Linux gets performance leaks when it is in need of services provided by other L4 services, since it then needs to call these services. One example for this is task creation, which I tried to improve recently. Let's first have a look at what L4Linux is doing upon a user-space application calling fork():
Having a look at what is done in task server and RMGR to startup a task shows, that it is simply some l4_task_new() system call. The first idea to improve the situation is to let L4Linux do so itself. I therefore added a new allocate_chief() call to the task server interface. This allocates a task for the client and makes this client become the task's chief. This is necessary, because only a task's chief is allowed to start and stop it.
This alone speeds up task creation and termination by around 10%, since we save one L4Linux -> task server -> RMGR call chain for both, starting and stopping a task. Another improvement can be made by caching unused tasks. If a task is terminated, we do not need to return it to the task server, but can put it into a cache instead. It can then later be restarted as a new L4Linux task. This removes every interaction with L4 servers from the start/stop process of a task. All in all new task management and task caching do result in 20% performance increase for task creation.
- Allocate an L4 task from the task server. The task server upon initialization got all available tasks from the resource manager (RMGR). However it does all these requests only to get informed about how many tasks are in the system. It then returns these tasks to RMGR immediately.
- Setup Linux-internal task data.
- Call the task server once again to start the task.
- The task server now calls RMGR to really start the task.
Having a look at what is done in task server and RMGR to startup a task shows, that it is simply some l4_task_new() system call. The first idea to improve the situation is to let L4Linux do so itself. I therefore added a new allocate_chief() call to the task server interface. This allocates a task for the client and makes this client become the task's chief. This is necessary, because only a task's chief is allowed to start and stop it.
This alone speeds up task creation and termination by around 10%, since we save one L4Linux -> task server -> RMGR call chain for both, starting and stopping a task. Another improvement can be made by caching unused tasks. If a task is terminated, we do not need to return it to the task server, but can put it into a cache instead. It can then later be restarted as a new L4Linux task. This removes every interaction with L4 servers from the start/stop process of a task. All in all new task management and task caching do result in 20% performance increase for task creation.
Saturday, May 13, 2006
[L4] L4Linux performance issues
L4Linux is a paravirtualized version of the Linux kernel running on top of the L4 microkernel family. For my dipoma thesis I am doing some evaluation of L4Linux in comparison to native Linux to find out where the former's performance problems come from.
One of the problems is that Linux applications are running as L4 user space apps alongside with the kernel. Every system call issued by an application leads to an IPC to the Linux server which then answers back once again using IPC. This means, that 2 context switches are invoked for each system call, resulting in a much larger number of TLB and cache misses than on native Linux.
I measured this by performing a nearly-null system cal: sys_getpid(). From Linux user space its average execution time is around 260 cycles on my test computer (An AMD Duron 800 MHz with 256 MB RAM, Linux 2.6.16 booted from a ramdisk). Performing the same task on the same computer with the same ramdisk setup in L4Linux results in around 3,700 cycles for each call to sys_getpid().
I thereafter counted cache and TLB misses for each setup and learned, that for 100,000 calls to sys_getpid() there were around 200 TLB misses in native Linux - probably from the point where my benchmark was interrupted by some other app. On L4Linux there were 6 TLB misses for each system call - some more. These misses lead to delay in execution of L4Linux system calls, because cached need to be refilled.
However, this is not the one and only source of performance leaks. Losing 3,500 cycles for a system call is not a lot, when you see that a blocking sys_read() needs 2,7 million cycles in average. There are other sources, for instance the points where L4Linux needs to use L4 system services to get work done. I will discuss this in another post soon.
Conclusion: Context switches for system calls reduce L4Linux performance.
Solutions already have been proposed:
One of the problems is that Linux applications are running as L4 user space apps alongside with the kernel. Every system call issued by an application leads to an IPC to the Linux server which then answers back once again using IPC. This means, that 2 context switches are invoked for each system call, resulting in a much larger number of TLB and cache misses than on native Linux.
I measured this by performing a nearly-null system cal: sys_getpid(). From Linux user space its average execution time is around 260 cycles on my test computer (An AMD Duron 800 MHz with 256 MB RAM, Linux 2.6.16 booted from a ramdisk). Performing the same task on the same computer with the same ramdisk setup in L4Linux results in around 3,700 cycles for each call to sys_getpid().
I thereafter counted cache and TLB misses for each setup and learned, that for 100,000 calls to sys_getpid() there were around 200 TLB misses in native Linux - probably from the point where my benchmark was interrupted by some other app. On L4Linux there were 6 TLB misses for each system call - some more. These misses lead to delay in execution of L4Linux system calls, because cached need to be refilled.
However, this is not the one and only source of performance leaks. Losing 3,500 cycles for a system call is not a lot, when you see that a blocking sys_read() needs 2,7 million cycles in average. There are other sources, for instance the points where L4Linux needs to use L4 system services to get work done. I will discuss this in another post soon.
Conclusion: Context switches for system calls reduce L4Linux performance.
Solutions already have been proposed:
- Processors with tagged TLBs do not need to flush their TLBs on context switches and therefore will reduce TLB misses for system calls.
- Cache coloring can be used to reduce overlapping between L4Linux and its applications so that both do not thrash their caches while running in parallel.
- Small address spaces are a concept to run multiple applications inside the same virtual address space. Thereby no context switching is performed for Linux system calls.
Wednesday, February 15, 2006
[Sports] Dynamo Dresden vs. TSV 1860 Munich
It's been a long time since we've seen some soccer in Dresden. The last game before Christmas (against Rostock) was postponed, because of heavy snow the days before. In January, season restarted and the first match against Fuerth also got delayed, because we had temperatures around -20°C and everything was frozen. With the away match in Offenbach also being delayed, Dynamo have not played three of their last four matches.
Last week the weather became slightly better, temperatures around the melting point. Dynamo officials tried everything to make Friday's game against Munich happen. Although the field does not have a heating, they pumped hot air through the drainage and thereby melted the frozen field. After this happened, it started to rain for some days and the field became a puddle of mud. Guess what - they had a helicopter fly around the field for an hour, thereby dry-blowing the grass. Crazy.
After the helicopter went away it started to snow again and this lead to about 50 fans cleaning the field allover Friday afternoon.
With all that said, the match nearly was a minor matter. Or not? Remember, the team had not won any of the last 13 matches. Therefore huge emotions burst out, when Thomas Broeker scored his second goal this season for the 1-0 before halftime. And better: we even got a second chance to make a goal and this was used by our new defender Tomas Votava to end up with a final score of 2-0 against a strong opponent, who could not manage to have any of their chances get countable.
Several good players were seen this day: keeper Ignjac Kresic saved the lead in half one and even got a concussion. Therefore he was replaced by Darko Horvat in half two and he also did an extremely good job. Best field player for me was Pavel Pergl, our new right-wing defender. I cannot remember when I saw one of our players win a one-on-one situation near the Dynamo goal this season. Pergl did this more than one time in a single game and was one of the reasons for Dynamo not to lose.
Last week the weather became slightly better, temperatures around the melting point. Dynamo officials tried everything to make Friday's game against Munich happen. Although the field does not have a heating, they pumped hot air through the drainage and thereby melted the frozen field. After this happened, it started to rain for some days and the field became a puddle of mud. Guess what - they had a helicopter fly around the field for an hour, thereby dry-blowing the grass. Crazy.
After the helicopter went away it started to snow again and this lead to about 50 fans cleaning the field allover Friday afternoon.
With all that said, the match nearly was a minor matter. Or not? Remember, the team had not won any of the last 13 matches. Therefore huge emotions burst out, when Thomas Broeker scored his second goal this season for the 1-0 before halftime. And better: we even got a second chance to make a goal and this was used by our new defender Tomas Votava to end up with a final score of 2-0 against a strong opponent, who could not manage to have any of their chances get countable.
Several good players were seen this day: keeper Ignjac Kresic saved the lead in half one and even got a concussion. Therefore he was replaced by Darko Horvat in half two and he also did an extremely good job. Best field player for me was Pavel Pergl, our new right-wing defender. I cannot remember when I saw one of our players win a one-on-one situation near the Dynamo goal this season. Pergl did this more than one time in a single game and was one of the reasons for Dynamo not to lose.
Friday, January 20, 2006
[Linux] Network transfer rates
For my diploma thesis I need to transfer monitored data from a test PC running the L4/DROPS operating system to my Linux workstation. A kind of weird idea of my tutor was to write all the data to a hard disk with no file system or so and then switch the disk to the other workstation and have a program read all the data. The good thing about the idea would be a high transfer rate. The bad thing is, that no real-time monitoring could become possible.
As I've been developing a software network switch for L4 last year, I thought that transferring data through a network could work great. For this I ported a small IP stack implementation, uIP, to use L4 and my network switch. Transferring small amounts of log data is no problem with it, but for large transfers we encountered the limits of this stack. Some of its small size results from minimal TCP features implemented and the drawback for large transfers is uIP being able only to send one TCP packet at a time and then waiting until the packet gets acked by the receiver.
On my Linux machine I started using netcat to receive all the data and store it to a file. What I experienced there was a relatively fast start of transmission (~20 kB/s in the first 2 seconds) and then a sudden slowdown to only 2kB/s afterward. Ethereal can really be your friend when diagnosing network connections and so I found out that the Linux machine started sending acknowledgements fast and after some time considered the TCP-stream to be one-way and only sent one acknowledge message every 0.5 seconds. This is okay for a normal TCP stack, because TCP allows acknowledgement of several packets with one ACK message. uIP however needs the ACK to send the next packet and therefore everything messed up and became slow.
After some time of discussion (with even considering the option of porting yet another IP stack to L4) and searching, I found that there is an option for TCP sockets, putting this socket into the QUICKACK mode, where all packets are acked immediately. So I abandoned netcat and wrote my own little socket application that puts its receive socket into this mode for each receive. This finally gave me data transfer rates of 2.2 MB/s - 1000 times better than before.
If I should find this throughput too slow in the future (monitoring produces a lot of data), I will port the lwIP stack to L4, but I hope this will not be until I receive my diploma.
As I've been developing a software network switch for L4 last year, I thought that transferring data through a network could work great. For this I ported a small IP stack implementation, uIP, to use L4 and my network switch. Transferring small amounts of log data is no problem with it, but for large transfers we encountered the limits of this stack. Some of its small size results from minimal TCP features implemented and the drawback for large transfers is uIP being able only to send one TCP packet at a time and then waiting until the packet gets acked by the receiver.
On my Linux machine I started using netcat to receive all the data and store it to a file. What I experienced there was a relatively fast start of transmission (~20 kB/s in the first 2 seconds) and then a sudden slowdown to only 2kB/s afterward. Ethereal can really be your friend when diagnosing network connections and so I found out that the Linux machine started sending acknowledgements fast and after some time considered the TCP-stream to be one-way and only sent one acknowledge message every 0.5 seconds. This is okay for a normal TCP stack, because TCP allows acknowledgement of several packets with one ACK message. uIP however needs the ACK to send the next packet and therefore everything messed up and became slow.
After some time of discussion (with even considering the option of porting yet another IP stack to L4) and searching, I found that there is an option for TCP sockets, putting this socket into the QUICKACK mode, where all packets are acked immediately. So I abandoned netcat and wrote my own little socket application that puts its receive socket into this mode for each receive. This finally gave me data transfer rates of 2.2 MB/s - 1000 times better than before.
If I should find this throughput too slow in the future (monitoring produces a lot of data), I will port the lwIP stack to L4, but I hope this will not be until I receive my diploma.
Sunday, January 08, 2006
Terry Pratchett - The Tiffany Aching Series
Terry Pratchett has written two books about Tiffany Aching up to now, which I finished reading last week. Tiffany is a young girl trying to become a witch. In the first book --- "The wee free men" --- she has to save her little brother (Who is always crying for sweets - reminds me of the abbot in "Thief of time") who has been stolen by the Queen of Fairies. Book 2 --- "A hat full of sky"--- tells about her apprenticeship with an elder witch. Thereby Tiffany is occupied by an evil demon who steals people's thoughts.
This may all sound a bit frightening, but it is however quite funny. The fun comes mostly out of the supporting characters. Tiffany makes friends with the Nac Mac Feegle - a clan of six-inch-high blue men always looking for a fight (and if there is none, they'll fight each other). These guys are quite funny (but not very bright...) and also had a small appearance in one of the real witch-novels, "Carpe Jugulum". Furthermore discworld's greatest witch of all --- Granny Weatherwax --- has got her appearances and is really part of the story in the second book.
The Tiffany-Aching-series is not as complex to read as other discworld novels, mainly because it is targeted at younger readers. (Yes, I read children's books!) However, it is really funny, apart from the first third of the second book being a little bit weak.
This may all sound a bit frightening, but it is however quite funny. The fun comes mostly out of the supporting characters. Tiffany makes friends with the Nac Mac Feegle - a clan of six-inch-high blue men always looking for a fight (and if there is none, they'll fight each other). These guys are quite funny (but not very bright...) and also had a small appearance in one of the real witch-novels, "Carpe Jugulum". Furthermore discworld's greatest witch of all --- Granny Weatherwax --- has got her appearances and is really part of the story in the second book.
The Tiffany-Aching-series is not as complex to read as other discworld novels, mainly because it is targeted at younger readers. (Yes, I read children's books!) However, it is really funny, apart from the first third of the second book being a little bit weak.
Subscribe to:
Posts (Atom)
