AireLinux: hack the universe

Project Aire Linux

1 Introduction

AIRE1 (Astrophysical Integrated Research Environment) is a set of computing facilities initiated by Tsinghua Center for Astrophysics 2, which aims to aid astrophysicists collect, store and analyze data3.

With the development of computer technology, it is feasible to make simple computation on desktop, even mobile computers. At the same time workstation and computation server becomes more financially accessible, which enables small research groups deploy their own commutating facilities.

The availability of software, however, is in no sense trivial as compared to hardware. Historically, tools of astrophysical research are written for and tested on POSIX environments, e.g. Unix, which calls for expertise in administration and maintenance.

AIRE has a user-ready software environment, powered by GNU/Linux, developed by astrophysical experts and power users. Developer write home-made setup and environmental setting scripts for user. The present approach doesn't have an overall guidance and quality assurance standard for deploying tools. As the installed software grows, the setup style differentiate. This inconsistency and arbitrariness results in two main drawbacks:

  1. It is hard to continue maintaining other developer's tool, when, e.g., original maintainer retired. It is often easier to deploy from scratch, which discourages sharing of effort.
  2. It is hard to reproduce the same software environment in other site, for independent users and groups. Most setup are done by developers manually, with one-shot throw-away scripts.

The natural question would be: How to make maintenance of AIRE easier, well-organized and more reproducible?

This project is going to answer this, by introducing a package management policy and distribution road-map.

2 People

This project is supported by THCA, the speaks person being Doctor Jianfeng Zhou(周建锋)4. AIRE chief system administrator is Zhuoxi Huo(霍卓玺). The technical support is from TUNA(Tsinghua University Network Administrators)5. Benda Xu(续本达) is in charge of this draft proposal. Qijiang Fan(樊栖江) and Liang Xu(徐亮) is on the board of draft advisory committee.

THCA will host this project. People officially involved will get funding from THCA. An army of developer from THCA, TUNA as well as software and/or astrophysical community will be employed part time and paid proportional to their effort.

Present AIRE users will be invited to evaluate deliverables of the project.

3 Goal

The outcome of this project is threefold:

  1. Astrophysical software packaged and consistently maintained in Linux distributions, at present chosen as Gentoo6 and Debian7.

    The list of candidate packages are:

    TODO (to be compiled by Zhuoxi Huo)

  2. A Linux distribution named "Aire Linux", based on Gentoo and/or Debian. We provide installer, LiveDVD/LiveUSB and virtual machine image, as well as installation guide.
  3. (Far future) Deploy and integrate a cloud computation framework in the distribution for end users to share computation resources to support astrophysics research.

4 Technology

4.1 packaging

Gentoo Portage8 ebuild9 and Debian dpkg/APT10 deb11 will be developed for each software in the present in AIRE server.

  1. ebuild and deb will be developed in parallel, sharing patches and experience.

    Gentoo is source-based meta-distribution12. An ebuild is essentially a set of standardized instructions for fetching, patching, compiling and installing the software. This is easy for packaging and hard to end user who do not have software compiling experience.

    Gentoo Prefix13 can be deploy as a second package system on almost all kinds of POSIX environment, which gives us a possibility of a new form of distribution. It could be deploy alongside with other systems. This non-invading style (no dual boot or virtual machine) encourages computation resource.

    Gentoo Linux is used in AIRE at present.

    Debian is a binary-based universal distribution. A deb is a set of pre-compiled files to be installed into a fixed location. It is hard to package and easy for end user to use, compared to Gentoo ebuild.

    Debian is the main distribution used on servers of TUNA. AIRE uses Ubuntu, a Debian-based distribution, on one of its servers.

  2. ebuild and deb will be pushed distribution upstream.
    Email from Benda Xu, Dec 2, 2012
    我觉得我们没有足够的经验和实力来做 Quality assurance。与官方一起工作一

    Pushing packages upstream boosts its chance to be tested by a broader audience.

    Work with Gentoo-science14 and Debian-science15 groups. We become package maintainer in the long run.

4.2 Distribution

Email from Benda Xu, Dec 2, 2012
采用 debian pure blend[1], 在 debian science 的基础上造一个 debian
  1. Installers, we use upstream installing methods, and guide user to set up astrophysical packages.
  2. LiveDVD/LiveUSB, based on Gentoo LiveDVD16 and Debian Live17. Serves as evaluation option as well as installation media.
  3. Hard disk images for virtual machines. Clone out disk images from an installation for virtual machine users.

4.3 artwork

The project leader (to be elected) is in charge of employing art professionals to designing project website, logo, splash screen, desktop wallpaper, etc.

4.4 Infrastructure

This project will be umbrellaed under TUNA, making use of its wiki18, roundup issue tracker19 and distcc compilation farms. Mailing list will be created as a google group20. Optional IRC channel can be opened on freenode21.