Hemisphere Games Banner

Porting Osmos to Linux: A Post-Mortem (part 2/3)

Welcome to the second installment of our three-part Linux post-mortem. Part 1 lay the foundations for the article and where we’re coming from. Today’s post directly addresses the question “What worked and what didn’t?” with a set of ‘pros and cons’ that cropped up during the port. The final post will offer a more nuanced set of reflections on these experiences, their implications for game development, and my advice for moving Linux forward as a gaming platform.

Porting Osmos to Linux: What I Loved

Let’s start with an account of the stuff that stands out as having worked really well — things that were well designed, pleasant to use, and easy to learn.

Love: Live discs and bootable USBs

Being able to boot into an OS from USB without having to first install to the HD is fab, and the various Linux distros have for the most part done a good job of supporting this behaviour. During development, live discs were helpful in two main ways. Firstly, when I was starting out, they made it easy for me to poke around and find a distro to use as my ‘core’ Linux install (i.e. the install on which to do the bulk of my Linux development; I settled for Ubuntu 9.04). Secondly, live discs made it easier to do quick first-blush tests of cross-distro/DE/WM compatibility for my executables and packages without having to first fully install and configure every OS.

One wish for the live discs would be for the OS versions they contain to have more robust HW drivers, particularly for video. I realize drivers are a tricky issue in Linux and hard to support generically out-of-the-box, especially on a live OS (where, for example, rebooting to install a better driver isn’t in the cards), but for game development it would be great to have decent video performance when booting from a live disc.

Love: Code::Blocks

I was looking for a simple, light-weight, easy-to-use C++ IDE that was easy to find in binary form for several distributions (I didn’t want to have to compile an IDE myself) and that had some kind of interface sitting on top of gdb. Code::Blocks was the answer. It wasn’t bug free, mind you, and crashed on me once or twice per day (at times to great frustration because of state loss… but nothing reliably reproducable, I’m afraid, hence no bug reports filed). But, critically, it provided a simple project system, a simple-yet-powerful set of compiler/linker options, the essential find/error/debug/bookmark interactions, and a basic gdb front-end. Essentially everything I needed was there, it was simple to use, and it was available and (relatively) stable cross-distro.

Love: The Linux Package Manager

Linux’s package system may not have been easy-breezy to support (more on this below), but it sure was easy-breezy to use. It was straightforward to find the developer libs/headers I needed and get them automatically installed into the right places. It was fantastic that dependencies are automatically tracked and installed, and that headers/libs install to central and standard places for trivial inclusion/linkage into the project.

Two footnotes: firstly, this smoothness of experience was conditioned in part by the legwork that went into ensuring Osmos relied only on the most core of libs (libs/packages that ought to play nice and be easily found on any distro; GL/GLX, X11, FreeType, OpenAL, libogg/libvorbis, etc). Secondly, a couple of the enterprise-oriented distros seemed to require a prohibitive amount of leg-work to configure for engineering purposes; this makes sense if the distro is targeted towards the consumer/office end of the market spectrum rather than the engineer, but for someone new to development on Linux, it’s painful if you don’t do your homework ahead of time.

Love: Support from the Linux Community and our Beta-Testers

It’s great to see how keen Linux users are to help support software on Linux. The average Linux user is on-the-ball and really encouraging in terms of their enthusiasm for Linux and their appreciation of our efforts to get Osmos on Linux. Post-launch, we’ve been getting great feedback and PR support from a few key folks in particular.

When the Linux port went into beta I issued a public call for testers and recruited about 50 people. Each one was top-notch in terms of being observant while playing the game. They were all helpful with their detective work, and the average level of technical savy-ness was very high when it came to sniffing out audio and WM/X issues for repro. Folks were keen to try the game on as many machine specs as they could, and were keen to volunteer information and “run with” the testing on their own. All this resulted in a very short beta period (less than two weeks) in which a lot of solid testing occurred very quickly, and with only a single patch being required to hammer out all known issues.

Love: Cross-Platform Compatibility is Good for your Code

While not a Linux-specific observation, there’s no doubt that developing multiplatform software is always beneficial for the code. Each platform has different analysis tools, compilers that issue different warnings and generate different code, and video drivers with different behaviour; all of these serve to highlight any underlying issues in your project, and so the work invested on one platform has benefits for the other platforms too. For the sake quality and stability, cross-platform compatibility has been a good thing for Osmos.

What I Didn’t Love

Let’s move on to the stuff that was less easy to love; things that stood out as being frustrating, difficult to use, and in some cases broken.

Didn’t Love: Supporting multiple Distros/DEs/WMs/drivers/etc.

The #1 obstacle to getting more games on Linux is that it’s very difficult to get your game working correctly and acceptably on all machines. It’s really hard to guarantee a smooth experience for all players when there’s a combinatorial explosion of possible distributions, desktop environments, window managers, driver/hardware versions — each with their own unique foibles, bugs, and undocumented behaviours. It’s the classic PC gaming problem, but magnified. Linux loves freedom and choice, which I applaud — but some amount of standardization, collaboration and “Let’s work together”-ness is required for the platform to be friendly to application developers.

Didn’t Love: Audio

Brace yourselves, folks, because I’m going to put it bluntly: audio in Linux is a mess. The audio situation is another major obstacle to game development on Linux. It’s 2010; audio is a solved problem. And yet on Linux, it’s shocking that you cannot count on something as simple as non-streaming playback, never mind any kind of processing (which most games are wont to do!) There are a variety of standards (ALSA, OSS, PulseAudio, Phonon, …) — which one to choose? Each standard has different problems on different machines. For a given standard, some drivers are buggy and poorly configured by default, while others do horrible thinks like block on open when another process opened the device. This is all another way of saying that there are no audio standards. What is a developer to do?

“But doesn’t OpenAL hide all the mess behind a single stable API? Ideally you’d write the audio code once in OpenAL and then walk away.” This isn’t how it happens in practice, because the fact of the matter is that the mess is still there. Simply wrapping another software layer around the underlying problem doesn’t make it go away. Using OpenAL for audio doesn’t all-of-a-sudden mean I can count on anything runtime-behavior-wise. I write my code in OpenAL (rather than having to write directly in ALSA, PulseAudio, etc), but at runtime I still need to make choices — for example, by saying which underlying audio layer I want to use — and different people’s machine/lib/driver combos will do different things. Simply using the local machine’s default audio device doesn’t guarantee anything about what’s going to work and what doesn’t, and a game needs guarantees in order to work correctly. The audio situation is quite horrible I’m afraid.

Didn’t Love: Lack of Documentation and Consensus

In the Linux world there is so much choice and non-standardization, and it’s really hard to find out about things. Documentation is sorely lacking, and it’s hard to find solutions because when a question is asked, people don’t agree on the answer. Forum threads aren’t a good substitute for proper documentation, because forum threads can quickly become historical and fall out of date, meaning folks looking for answers spend a lot of time chasing down false-leads and asking themselves “Is this forum thread relevant to me? Is it really what I’m looking for?” In the Linux world, it seems to take a great deal of detective work and reverse engineering to get things done; the plethora of choice means that the newcomer is never certain about their choices — and there will always be someone who disagrees with you (often vocally).

Didn’t Love: Drivers and Hardware Support

My main development machines are two off-the-shelf laptops with mainstream components, and neither of them had WiFi or video that worked out-of-the-box on any distribution I installed. It took many hours to get decent hardware performance. Proprietary drivers, open source drivers… which do I chose? Do I really have to compile a driver for myself? What’s more, when a new Linux user inquires about driver problems on a forum, the standard line of defense is to blame the problems on hardware manufacturers. Whether or not the manufacturers deserve blame, the Linux platform folks need to step up, acknowledge the reality of the situation, and try to work to improve the platform. The chipsets that didn’t work for me (without great effort) were nowhere close to bleeding edge (ATI and Intel graphics, Broadcom Wifi; some models dating back to 2007) and all are widely available on a large segment of the consumer-level market. Linux needs better support for its HW; if I can’t walk into BestBuy, pick up a run-of-the-mill laptop and have my video and networking hardware work, then the platform is troubled.

Didn’t Love: Packaging the Game

It took days of effort to create the binary packages for Osmos (I’m happy to say that it’s looking like we don’t have to patch!) How should an app be packaged in Linux? Should I build my own libs and package them with my game, or rely on package dependency info and hope the distros have the right versions in their package repositories? Which package formats do I choose (.deb, .rpm, .tar.gz, others)? Do any of the package formats have naming conventions I need to follow? What do you need to do to support both 32bit and 64bit? What are the standard practices for where/how my game is layed out on the filesystem? What goes where and where do you commonly put softlinks? How do I represent softlinks in the various package systems? Do architecture-specific files belong in special places? Are there any standards for what’s in the environment path? How do I integrate my app into the desktop environment? What DEs are worth supporting? Do I need to do separate things to support separate DEs?

There are no standards or clear answers to any of these questions. There’s no documentation for this stuff! Asking on the forums will typically net you a spectrum of answers with no consensus answer and lots of little side arguments. I basically reverse engineered what I saw other apps doing (which sadly was of little comfort because everyone does it differently). I settled on supporting .deb/.rpm/.tar.gz with explicit 32bit and 64bit executables for both the demo and retail versions of Osmos, with no redistributed libs and instead relying on package dependency info. So far, this has worked out for nearly all distros except CentOS which has an archaic version of libvorbis and nothing new in the standard repositories.

Side-note for those interested: I didn’t venture anywhere near cross-compilation and instead simply built the 32bit and 64bit executables on separate 32bit and 64bit Ubuntu installs (which, interestingly enough, both displayed unique and undocumented WM/Xserver interactions, despite being the same version of Ubuntu!)

Didn’t Love: No OS-level GUI layer for simple dialogs

This is something of a minor point compared to the above, but I want to mention it because it comes up often enough in cross-platform development. Because Linux has no OS-level GUI layer, games that need any kind of UI must link against heavy-weight UI libraries (GTK, QT, etc) which typically impose some kind of application framework. Common examples of the usage of UI in the gaming world would be a dialog that prompts the user for input the first time a game is run (e.g. “Launch in fullscreen?”) or that displays a message when an app terminates unexpectedly. For Osmos, I had to cut such user-friendly elements because I didn’t want the game to have any but the most basic of dependencies.


Whew! Alright: in part 3, I’m going present a more general discussion of the implications of some of these issues for game developers, and reflect on ways to improve matters. Stay tuned…

Dave

45 Comments on “Porting Osmos to Linux: A Post-Mortem (part 2/3)”

  1. #1 Kyran
    on May 18th, 2010 at 4:16 pm

    Very nice post.
    First: I totally agree on the audio part. I’m always amazed any sound is produced at all. (I also had a nice low latency setup on ubuntu karmic, but upgrading to lucid totally broke that, for no reason)

    As for the no os-level gui layer: the kde desktop environment and ubuntu have implemented a notify spec, which I think is pretty lightweight and leaves all the drawing of the notification to the desktop environment. Of course this still leaves anyone not running kde 4+ or ubuntu out in the cold, but they’re pushing hard for this to become a standard.

  2. #2 Jon Pritchard
    on May 18th, 2010 at 4:33 pm

    It’s very interesting to hear your experiences. I’m not a game developer, just an end user and even I encounter some of the problems you have.

    I’ve been trying out a lot of Linux ports of games recently (from the Humble Indie Bundle and others) and I’ve not had good out-of-box experiences with many. This really highlights the issues you put forward. In packaging, dedicated or system libraries etc. and hardware configuration, including audio.

    On the more general Linux side I’ve been lucky that all hardware I have tried has worked very well under Linux. Although I’m about to install on a desktop computer and the large array of supporting chips like SATA controllers, worries me a bit.

    I thank you for porting your game. I saw it on Reddit just after you’d done it. I’ve still not got around to trying out the demo yet, but hopefully I’ll be able to make a purchase decision after that (not had much time to play games right now, and I’ve bought a lot recently).

    I look forward to the next instalment of your article.

  3. #3 Jon Pritchard
    on May 18th, 2010 at 4:35 pm

    Oh I just have to say as well. Thank you for allowing customers to use all the different game clients. I bought the Penumbra Collection recently and was not too happy that I couldn’t play it on Windows if I wanted to.

  4. #4 Apopas
    on May 18th, 2010 at 4:54 pm

    I never understand the packaging troubles. Maybe each distro has its own package system, deb, rpm ec etc but all support the sh and bin files. That’s what Loki, LGP and the other linux game manufacturers use.

  5. #5 Steven
    on May 18th, 2010 at 5:00 pm

    Thanks for writing this up, it was an interesting read. Regarding simple dialogs, you might have missed zenity. It is a simple standalone app that you can use to display various informative and interrogative windows. Since it is a standalone aapp it would save you from having to add extra link dependencies to osmos itself.

  6. #6 Mr. X
    on May 18th, 2010 at 5:20 pm

    “especially on a live OS (where, for example, rebooting to install a better driver isn’t in the cards), but for game development it would be great to have decent video performance when booting from a live disc.”

    Use the option to save your OS changes to the USB instead of having it boot from new each time. USB Startup Disk Creator/UNetBootin both give you those options.

  7. #7 ~:SahilShines:~
    on May 19th, 2010 at 8:20 am

    great post man!!!i am being with linux for couple of years but didt know about dese problem….thnx
    dis will help me as meh to undertaking animiation courses …..
    i hope linux solves d api and audio stadard problem….
    LINUX IS HUNGRY 4 GAMES…..BEST OF LUCK

  8. #8 ~:SahilShines:~
    on May 19th, 2010 at 8:22 am

    SORY MIS SPELT…m undertaking animiation cources….sory poor english
    keep up ur hard work…m gon to play ur game,video is great

  9. #9 mivoligo
    on May 19th, 2010 at 8:52 am

    Many thanks for the game and this article!
    Are you planning to port Osmos to Nokia N900 by any chance?

  10. #10 Joseph
    on May 19th, 2010 at 11:21 am

    You really really need to talk with Jono Bacon (Canonical employee; Ubuntu Community Liason) or Shuttleworth. Basically, your points seem to boil down to hardware support (we’ll get that in the next paragraph) and needing a contact to tell you what libs/interfaces to use and a responsive bug reporting contact. Jono is definitely your man for the software side. At worst, he can put you in contact with the developers you need to talk to and Canonical personnel who can help support ISVs such as yourself. In addition, you can market your game through Canonical’s store to hopefully sales.

    The hardware problems boil down to as much a lack of understanding as the people who ask why you don’t just use SDL. Perhaps Apple is smarter in this regard by taking away your freedom to install OSX on whatever hardware you choose. I think you’d be very, very surprised at how non-mainstream mainstream hardware really is (which is definitely biting you with the drivers issue and almost certainly on Pulse; audio hardware often is wrong and/or lies, and pulse relies on this information being correct; there are a number of blog entries on this (that I can’t seem to find now that I need to link to them. Argh!) This leaves out the myriad of OEM tinkerings e.g. with the VBIOS and vendors that are actively Linux-hostile like Broadcom). The point is that it’s great that Linux works as well as it does on hardware that was Designed for Windows. Like with Mac, if you want the easiest Linux experience, you really should buy a Linux computer, e.g. from System76 or ZaReason (these come with Ubuntu pre-installed; ZaReason offers some more distros if you so prefer). You can get very nice hardware with Llnux pre-installed for between $300-$600 (very high-end nettop or a low-end desktop) which should be more than adequate for your Linux development needs. Plus, then you don’t have to play the OEM’s role of system integrator (making sure that all the hardware works with the software); they do that for you. Plus you can call up their support and get support for Linux/Ubuntu on their hardware (you can get in-depth Ubuntu support if you buy a support contract, just like with Windows you have to pay MSFT to get anything more than Windows-on-Dell if you call Dell.)

  11. #11 Joseph
    on May 19th, 2010 at 11:22 am

    (“hopefully sales” should read “hopefully increase sales” :)

  12. #12 Joseph
    on May 19th, 2010 at 11:24 am

    (and to complete the parallel with SDL; it looks great for your use from high-up, but it falls down for your needs when you get down to the nitty-gritty details of what you’re actually doing. Similarly, it’s easy to say “Why doesn’t Linux support mainstream hardware! It’s 2 years old” but it’s much more understandable when you get down into the details of what’s gong on.)

  13. #13 Mackenzie
    on May 19th, 2010 at 1:33 pm

    Someone didn’t read the Debian Policy Manual if they couldn’t figure out how to list dependencies in a package, what naming conventions are (use – not _ in the package name, all lowercase, etc), and that a single source package can be built into both 32bit and 64bit packages.

    Where do files go? You don’t think about this. The package handles it. Just run “dh_make” and it’ll figure it out for you.

    As to integrating with the DE? The minimal is just to make a .desktop file which lists the proper category for it to go in the menu. It’ll automatically enter that section of the menu in whatever DE is in use.

  14. #14 Meal Worms
    on May 19th, 2010 at 2:43 pm

    @ Kyran: Cool beans regarding the notify spec, but tying an app to a single distro/DE and therefore slicing your user base doesn’t really fly. Hopefully this initiative or one like it will become a standard!

    @ Joseph: Fair point regarding the challenge of properly supporting HW, and I acknowledge it’s not an easy problem! It’s just a reality that in my mind needs to get solved for Linux to really take off in the desktop/laptop space. If the HW is designed with Windows in mind, then how about a layer emulating the way Windows talks to hardware? It seems that Linux market penetration will be limited if having a pre-built Linux machine is the only way to ensure a smooth HW experience. This is especially true in the gaming space where there are a wide spectrum of configs, particularly for graphics; we need to ensure a quality of experience for as wide a range of players as possible.

    @ Mackenzie: Thanks for the advice, but you’ve just underscored my point, which is that these things should be easy to figure out *but aren’t*! To simply say “Oh, you obviously looked in the wrong places; you should have done X, Y, and Z” doesn’t fly, because yours is only one in a long line of conflicting suggestions about best practices for building a binary package, arranging an app, etc (and FWIW the developer is still responsible for the application’s filesystem layout; dh_make doesn’t help you here).

    My point is that everyone, like you, has their own ‘best practices’ for doing these sorts of things, that can only come from experience because the documentation is lacking, scattered or dense (no quick starts, lots of forums and out-of-date-ness). Now multiply this learning investment by N package versions, M DEs and whatnot, and you have an obfuscated and complex learning process which poses a significant barrier to entry to application developers who aren’t even convinced the platform marketplace is worth their while in the first place! And we’re only talking about packaging the app, nevermind the port itself!

    Dave

  15. #15 Joseph
    on May 19th, 2010 at 3:51 pm

    “It’s just a reality that in my mind needs to get solved for Linux to really take off in the desktop/laptop space.”

    The long-term solution is for people to demand hardware that easily works with more than just Windows. It’s rather a chicken-and-egg problem. Like “Just use SDL!” it is extremely to say “why don’t you just do X” without actually looking at the nitty-gritty details that are where the rubber actually meets the road. Sometimes the dune really is just the sum of the grains of sand.

    “If the HW is designed with Windows in mind, then how about a layer emulating the way Windows talks to hardware?”

    This is already done as much as possible, bug-for-bug. See also Linux’ ACPI support. Unfortunately, you’re asking for the hardware equivalent of WINE. With this parallel, you can perhaps see here the scale of the difficulty involved. Particularly when the entity you’re emulating has a long anti-competitive history of changing things under their control to fend off overly ambitious competitors and which has recently been throwing its patent weight around (which is a whole ‘nother kettle of fish entirely).

    “This is especially true in the gaming space where there are a wide spectrum of configs, particularly for graphics; we need to ensure a quality of experience for as wide a range of players as possible.”

    We’d love to do just that. Unfortunately, the reality of the situation is that people (both consciously and unconsciously) buy hardware that is broken in a wide variety of ways but yet which works (most of the time!) in Windows. Until either Linux can somehow be *exactly* like Windows bug-for-bug for each Windows release (being careful, of course, to not do the wrong *version* of Windows on the wrong piece of hardware!) and replicate all of the third-party drivers and their kludges and workarounds and bugs *or* until people start buying hardware that works with Linux in sufficient numbers that even Broadcom (well-known for being hostile to Linux) has to put out hardware that works with Linux, you’re gonna have your situation recurring again and again. We’re doing what we can, but it’s nigh on literally a sisyphean task! I’m sure you can have quite a conversation with Canonical about this when you get in touch with them.

    The short-term *and* long-term solution for each individual user who wants to see Linux Just Work on their PC is to buy Linux PCs, just as they do for Mac and Windows.

    “the documentation is lacking, scattered or dense (no quick starts, lots of forums and out-of-date-ness) Now multiply this learning investment by N package versions, M DEs and whatnot, and you have an obfuscated and complex learning process which poses a significant barrier to entry to application developers who aren’t even convinced the platform marketplace is worth their while in the first place!”

    This is where you’d really benefit from talking directly to canonical. I assume you talk to Microsoft (perhaps via an MSDN subscription) and Apple for your Windows and Mac information?

  16. #16 Nils R Grotnes
    on May 19th, 2010 at 5:05 pm

    FLOSS usually starts slow (“scratching your own itch”) but the openness means it can sustain growth in a way that makes the good stuff really take off sooner or later. You describe how your beta-testing worked very nice, but seem to have taken upon yourself to do all the legwork around searching for information and documentation. Maybe a more FLOSS-like model could have helped, why not host something like a Wiki for game developers interested in Linux? (The “scratching your own itch” again…) Others may be spared from having to go through the same steep learning curve you did, and with a bit of luck (and PR) you might trigger the same helpful response you got for the beta testing. In my opinion, collaboration is the key to releasing the full potential of FLOSS.

    Drivers is a well know problem, and there’s many smart people working on that problem. This link documents some of that work: http://www.linuxfoundation.org/collaborate/publications/linux-driver-model

    As for sound, it’s true it is in a state of turmoil right now, but personally I think it is temporary. Regularly older designs are scrapped, and better ones developed. In the time I’ve been using Linux, this has happened for both graphics and later wireless. Wireless might still not be perfect, but compared to just a year or two ago, it has improved impressively. Of course the new sound stuff might still not support the needs of games, but that would at least in part be a result of game creators not using a bit of time to participate in the work themselves.

    It can’t be said enough, the price of FLOSS is not zero, it’s contributing back where you can. This post is one tiny bit of such a contribution, just as your “post-mortem” seems to me to grow into a very useful bigger one. I just hope game creators also contribute some of their strength as programmers, at least by reporting bugs, but hopefully also with patches and code.

    Looking forward to the third post!

  17. #17 WalterCool
    on May 20th, 2010 at 3:16 pm

    About the audio thing… pulseaudio should be fine, is a great audio manager and works on KDE,LXDE and Gnome, should be a standard for desktops, i cant approve things like phonon (QT specific) or ALSA (is the global system audio! Pulseaudio works per-user).

    Dialogs = GUI = desktop specific… i know, i have a special class for that, im trying to detect “GTK QT or TCL/TK”, and im just creating GUI objects (of gtk, qt, tcl/tk) and calling it with the same name… doing that, my apps are supporting alternatives.

    Drivers and Hardware support is a big fault of manufacturers, not Linux… i hate that, they have good developers but bad hardware (broadcom), so, showing this code will kill the business (they are anti-competitive)

    Packaging… yes, is a big lack… i love deb packages, but many distributions are using rpm too, and people maybe love rpm…, im sorry, i hate binary installers, because package managers are doing a “order” and binary installers arent helping with that… For closedsource games, a binary (tar.gz maybe is hard for ‘users’) installer, a deb and a rpm.

    About multiple distros, just be aware of that… you are doing a game, if a Linux Distribution have problems with your game, just unsupport that distribution (yes, like CentOS… is stupid installing a game on a ‘server specific’ distribution… bad oriented), working on “common user distros” (TOP10 on http://www.distrowatch.org) should be fine.

    Thanks again for your Linux client of Osmos (if you can fix the known bug of “random freeze” should be better), i hate when a game needs a specific SO (Enterprises like EA or Activision/Blizzard, high demand of Linux users, but they are installing Windows “just for play”)

  18. #18 Raphael
    on May 20th, 2010 at 3:28 pm

    Thanks for the insightful post.
    Waiting for the final part.

  19. #19 M
    on May 20th, 2010 at 6:58 pm

    Thanks for porting Osmos! I’m sorry to hear that you had problems. Your best bet for answers to these questions is the Linux Standard Base. Not everything is in there, but following the LSB means that your customers only have to `$package_manager install lsb`to get things working. In particular:

    – Package everything in per-arch RPMs. The LSB documents
    – FS layout
    – package naming convention
    – command-line tool availability
    – ALSA for sound
    – Qt 4.x is the GUI toolkit
    – X extensions are documented (for fullscreening, etc.)

    The hardware problem is real, and Greg Kroah-Hartman has stepped up (http://www.kroah.com/log/). To be fair, the Broadcom wifi fiasco has been widely documented — not much can be done to improve drivers when a vendor openly antagonizes the community.

  20. #20 Gergely Kis
    on May 21st, 2010 at 4:41 am

    Hi,

    Could you post a comparison how long it took to port the application to other platforms (in particular Mac OSX), and what were the biggest issues there? I am particularly interested in:
    - how many developers worked on each port, and how much experience they had with the platform.
    - how many games ported your company to that platform before
    - how much of the platform specific plumbing could be reused in this project from the prior projects.

    Of course the points you make are unfortunately valid, and the Linux developers need to work hard to solve them.

    Best Regards,
    Gergely

  21. #21 glezmen
    on May 21st, 2010 at 4:44 am

    Thanks for your work and sharing your experiences

    about the audio: you’re totally right, audio in Linux is a HUUUUUGE mess :( i was fighting with a strong audio delay in my last game (using SDL), and finally it turned out that it’s because of PulseAudio, on my handheld game console and other distros it runs perfectly – and this bug is well known, without any fix…

    packaging: if you’re planning to write more multiplatform apps, you should consider using CMake, which will work on all platforms, and with CPack you can create packages in many formats with little effort (deb, rpm, tgz, even .sh)

    GUI: if the only thing you need is a simply popup before starting, you can use Xlib (which is a real pain, but for a simple dialog it’s not SO much) or even terminal with the ‘dialog’ utility which should be there in almost every systems, or maybe XDialog and add it as dependency (it’s really small, not as huge as GTK/Qt)

  22. #22 Geri
    on May 21st, 2010 at 6:22 am

    I very enjoyed reading your article. But developing games to linux is FUN! If you found your path to reach your goal. I has too a shorter article from game developing to linux, its in hungarian, but probably can read in google translator:
    first: legendgrafix.buzz.hu/archives/2009/01/09/A_linux_es_a_zart_forras/
    second: legendgrafix.buzz.hu/archives/2010/05/15/Windowsos_jatek_linuxra_portolasa/

    To dodge the most problems, i do the folowing:
    -i link every needed library statically – no dependencys
    -i use simple .zip file as package
    -my softwares must not installed, after extracting the zip, it needs simply to double click on executable, and it starts

    The librarys under linux are not more twisted than under windows. But who are alreday knowing the windows developing, need to know a very different system when he/she add linux support. I am very happy if i see that somebody releases a software for linux, i hope my links are understandable with google translator so it can help you in some questions!

    greetings: Geri

  23. #23 Dave
    on May 21st, 2010 at 4:32 pm

    I love the fact that you took the plunge into the abyss of Linux documentation and still built your game. I am sad to say that Linux is scattered in terms of standardization and documentation. However, I would enjoy to see how many download requests you have gotten for the Linux clients. The Linux community is fanatical about such things, and I hope your business thrives on it. I intend to buy this game in order to support further development on the Linux platform for more games.

    Cheers,

    Dave

  24. #24 Ulrik Mikaelsson
    on May 23rd, 2010 at 8:37 am

    Thank you for a great recap. I agree with all your points. Just a few reflections;

    Despite being a Qt fan myself, I think wxWidgets is what you’re looking for. Small, very portable (you could even use for windows/mac-ports), and few dependencies itself. (And, I think, somewhat integrated in Code::Blocks?). Just a tip for future projects.

    Regarding CentOS, though, I think it’s pretty safe to say that nobody is really surprised to have games not working on a distro with many-years-old base-versions. It’s aimed at stable servers, and great at that, but it’s really not a gaming-friendly distro, or even much of a Desktop-distro. (Except corporate environments where users are likely not even allowed to install your game anyways.)

    Regarding diversity/fragmentation, it’s a strength, and a problem. However, I think you’d do fine just focusing on a few of the top-distributions. (As seems you’ve done, though not explicitly stating it.). Basically, [Ubuntu, Mint, Debian] + [Fedora, OpenSuse, Mandriva, PCLinuxOS] would probably cover >90% of Desktop-installs, and fairly easily be supported in two package-sets. (Just as you seem to have done). Also, noone seems to have mentioned FreeDesktop.org, which is a reasonably supported standard among top-distros and DE:s, and covers most of the “distro-de”-integration that you would need. (Application-launchers and such.)

    Personally, I’m off to buy my second copy of the game, for my Girlfriend, who also liked it. I would like to point out, that despite of my support for Linux-gaming, that would not have happened without the excellent Demo you also released. Keep up the good work.

  25. #25 Vadim P
    on May 23rd, 2010 at 8:54 am

    I do believe there is an OS-level (or well, omnipresent), but I don’t remember it’s name. HoN and I think some mojo installers make use of it. Looks pretty fugly, though.

    Right-on review.

  26. #26 RyanT
    on May 23rd, 2010 at 7:01 pm

    Here’s some info you may have missed or not have gotten on the audio side:

    http://0pointer.de/blog/projects/guide-to-sound-apis.html

    http://0pointer.de/blog/projects/when-pa-and-when-not.html

    It seems to me like Linux audio is only a mess if you include every possibility as an equal, and not how they actually are (ALSA/Pulseaudio and OpenAL is pretty it).

  27. #27 RyanT
    on May 23rd, 2010 at 7:27 pm

    Some things that I find a little interesting to point out.

    In most cases, various distributions have established process to get a programme packaged for their distro and in what way. In the Free Software world, this is very doable – with outside contributors and people from the distros themselves able to take and package the software themselves, this may help relieve the stress of handling such things on their own. Compare this to proprietary development, where its you guys going it mostly alone in doing the packaging and making sense of different methods suggested.

    Same applies to a certain extent for distro specific problems, alongside the general code review and improvement suggestions you’d get. If your game didn’t work on a distro? Someone could provide the fix themselves. A method to make it work better cross-distro out of the box? Someone can do that directly.

    The Linux world makes more sense from a free software/open source view, as the work load can get shared out. When its you guys figuring it out on your own and from lack of prior experience, then it works against you a lot more.

  28. #28 Sunny Kalsi
    on May 24th, 2010 at 2:38 am

    M’s Comment (http://www.hemispheregames.com/2010/05/18/porting-osmos-to-linux-a-post-mortem-part-23/#comment-11489) is correct. Basically the LSB is where you go to find out what you should do to support a “standard” linux.

    I hate how people still mention OSS. It’s old, it’s off the table. Alsa is how you get access to the sound-card. Pulse-audio is per-user, and mostly used if you are an “application” (so, for watching videos or listening to MP3s), and if you expect to be running in Windowed mode there’s no problem writing for PulseAudio, but basically you can’t go wrong writing for Alsa. Configuring audio can sometimes be a pain for a user, but from a programmer’s perspective that doesn’t change the answer here. Alsa is built to share the sound card, so there are literally no issues.

    Packaging is literally an exercise in “read the LSB” or in Debian’s case, using the bevy of packaging tools to “do what the LSB says”. With UI you literally cannot go wrong if you choose either Qt or Gtk. If your app is written in C, use Gtk; if C++, use Qt. Most people have both installed, and the libraries are generally very “friendly” when it comes to co-existing.

  29. #29 ChaosEagle
    on May 24th, 2010 at 3:34 am

    Thank you for the insightful post-mortem so far. I’m no developer, so I wasn’t fully aware of the potential problems with developing games for linux.
    But I guess things get easier with every project you port to linux.
    Anyway, this post was the last straw for me to buy osmos for linux despite of the the tons of other (linux)games I have bought recently. That, and the fact that osmos runs perfectly on my modest ubuntu-driven netbook :)

  30. #30 Mountain Man
    on May 24th, 2010 at 9:01 pm

    WalterCool: ‘Thanks again for your Linux client of Osmos (if you can fix the known bug of “random freeze” should be better)…’

    O.K., so it’s not just me that’s having this problem. It’s very annoying. Great game, but this bug is almost serious enough to be a deal breaker.

  31. #31 Tommy Brunn
    on May 27th, 2010 at 3:05 am

    While I agree with most of your points, you must have been incredibly unlucky to not have neither video or wireless networking working on two machines. I’ve been using Linux for years, and I’ve never had a piece of hardware (except for a fingerprint reader, but I don’t really care about that) not work – despite not really looking into Linux support before buying. I’m especially surprised that you didn’t have support for your intel video chip, since I know for a fact that there are open drivers available by default for them. Broadcom is a known problem, but that’s a whole ‘nother (ridiculous) story.

    I’m glad you decided to port the game despite running into some difficulties!

  32. #32 Meal Worms
    on May 28th, 2010 at 12:27 pm

    Thanks for the comments all! The third post is in the pipe, promise! ;-)

    @ Waltercool :
    @ Mountain Main :

    Regarding the ‘random freeze’ bug: looks like there’s a known issue in the Linux kernel where frequent gettimeofday system calls can cause a freeze. Sigh. Sadly I cannot repro the freeze myself, but I hope to have a workaround to roll out for test shortly.

    Stay tuned…
    Dave

  33. #33 My Th
    on May 29th, 2010 at 5:22 am

    Sunny Kalsi (#28) ++

  34. #34 Meal Worms
    on May 29th, 2010 at 2:51 pm

    @ Sunny Kalsi
    @ My Th :
    Regarding OSS — a few comments:

    1) How are programmers new to the platform supposed to know OSS is out of date?

    2) You say that Alsa is the way to access the card, but all three of my machines were experiencing glitches with Alsa (three separate distros)… If Alsa is supposed to be the audio silver bullet, it ain’t working right ;-)

    3) Regarding UI: as I mentioned in the article, Qt and Gtk are massive libraries that impose framework-level restrictions on core application structure, and are not suitable for games. All other major platforms have lightweight UI APIs that are *libraries*, not frameworks.

    4) If LSB is the place to go for Linux ‘standards’, then the fact isn’t well-advertised. Ditto the packaging experience! Your reactions are typical of experienced Linux developers who have no perspective on what the platform looks like for newcomers! It’s crufty, cumbersome, standard-less, buggy, and poorly documented.

    Dave

  35. #35 Licaon
    on May 29th, 2010 at 7:04 pm

    OSSv3 that has gone obsolete in the kernel might be dead.
    But OSSv4 is alive and well: http://www.opensound.com/wiki/index.php/Main_Page not to mention the brighter future: http://4front-tech.com/hannublog/?p=36 ;)

    At least in my case OSSv4 is superior to ALSA, i got one too many sound pops and sound interruptions about an year ago and since the ASUS XonarDX support just got in I tested it and I never looked back.

    Regarding sales figures i guess it will go like this: http://blog.wolfire.com/2010/05/The-state-of-Mac-and-Linux-gaming ( see the part with “So why don’t the blockbuster games sell?” ) since Osmos has this late Linux release problem.

    Cheers

  36. #36 RyanT
    on May 30th, 2010 at 10:59 am

    Should note: I am NOT a developer. I used a mixture between Google, common sense and 10-20 minutes of my time looking around to find those resources.

  37. #37 DavidW
    on Jun 1st, 2010 at 2:44 am

    Regarding a lightweight way of displaying dialogue boxes, you might try FLTK:

    http://www.fltk.org/
    http://en.wikipedia.org/wiki/FLTK

    It’s allegedly pretty small, and usually statically linked (hence isn’t a dependency). It doesn’t look as nice as Qt/GTK though.

    I should admit that I haven’t used it myself much, so there might be all kinds of other pitfalls I don’t know of.

  38. #38 Søren Hauberg
    on Jun 4th, 2010 at 5:10 pm

    Interesting read. But what happened to part three?

  39. #39 j_at_chaperon
    on Jun 11th, 2010 at 9:24 am

    On the “love” side, I would add valgrind. This tool is amazing and has no true Windows equivalent (there are profilers but they somewhat perform lesser). Anyway, this article exactly matches my own experience (been in the video game industry as a developer for years).

  40. #40 Jason
    on Jun 12th, 2010 at 12:32 pm

    For the most native X toolkit, there is always xaw;

    http://en.wikipedia.org/wiki/Xaw

    As a user, I’ve only rarely ran into sound problems with Linux. But then I had almost always had my Audigy 2, unfortunately with my latest motherboard upgrade, there is too much noise on the PCI bus or something, ’cause it to crackle, no matter what OS I use.

    Onboard audio hardware just sucks, and the Linux and Windows drivers do as well. Sadly, I think that’s where most of the problem lies.

    The same thing with the Broadcom wifi. I can get mine to connect from my laptop, but as soon as I start any streaming media sites, it dies horribly. I have to use the hardware wifi off switch, then turn it back on or it won’t reconnect. This is a driver issue, I’m sure.

    Never had an issue with my Intel WiFi though.

    Great game by the way, I’ll be buying it when I get paid next. :D

  41. #41 eddybox
    on Jun 24th, 2010 at 2:14 am

    Thanks for the feedback everyone!

    As for part 3, we decided there wasn’t much more to add. That, and the guys from Shot Of Jaq recently did a great job of summing up the situation and suggestions for the future of game development on Linux:
    http://shotofjaq.org/2010/06/the-indie-revolution/?utm_medium=laconi&utm_source=twitterfeed

  42. #42 Joey
    on Jun 30th, 2010 at 6:22 pm

    Re the gettimeofday hang, a strace of Osmos when this occurs looks as follows. I have reproduced the bug 3 times in 1 day, I think twice was on Ambient Impasse 1 when merging with a quite large blob. Would be happy to try a fix.

    gettimeofday({1277939695, 694232}, NULL) = 0
    gettimeofday({1277939695, 694307}, NULL) = 0
    gettimeofday({1277939695, 694383}, NULL) = 0
    gettimeofday({1277939695, 694458}, NULL) = 0
    gettimeofday({1277939695, 694534}, NULL) = 0
    gettimeofday({1277939695, 694611}, NULL) = 0
    gettimeofday({1277939695, 694687}, NULL) = 0

    kernel 2.6.32 (Debian)

  43. #43 Mackenzie
    on Jul 1st, 2010 at 3:31 pm

    Except that’s not just me. The Debian Policy Manual is THE RULES for doing it on Debian. If you do it any other way, it won’t get into Debian. That’s the definitive guide. Period.

    And the FreeDesktop.org specification I linked to? Again, definitive. That is THE standard across desktop environments.

  44. #44 Quora
    on Feb 15th, 2011 at 12:45 pm

    Linux: What can be done to make Linux better for application developers?…

    Hemisphere Games has a good list of developer pain points in Porting Osmos to Linux: A Post-Mortem (part 2/3)…

  45. #45 Fairy Charms
    on Mar 10th, 2013 at 3:28 pm

    hello there and thank you for your info – I’ve certainly picked up anything new from right here. Fairy Charms http://45gh.com/plus/view.php?aid=15211

Leave a Comment