Home close (×)
Attention:   The information you find below is outdated

Cool threads and hot nerves

Commissioning Part 2: Linux, netboot (cont'd)

I went for 'Finish partitioning'. For some reasons the creation of the all the file systems took about 1.5 minutes; normally XFS and swap under Linux/i386 are created within a fingersnip.
 The Ubuntu installer then prompted for the timezone & clock and a user had to be created. For one user there were three dialog boxes drawn in ncurses characters over the slow serial line. The question popped up whether one screen would not be enough. Over the next one and a half hour according to the progress dialog window the base system was installed, the installer configured apt, prepared the language pack (language-pack-en-base) and 'selected and installed' some other software within the next 40 minutes. It probably didn't have the intelligence built-in to check whether there was a graphics card, since it was prompting to configure the X-server – 'fbdev' is the right choice here. Some other packages were 'selected and installed' and 'preparing to configured', before (attention: irony) the installer remembered that it forgot previously to ask for the screen resolution which I saw 5.5 hours later on the screen. As one can imagine, I was not paying attention the whole time, so take these time informations with a little grain of salt. Then it 'prepared', 'unpacked' 'configured', software for another 2.5 hours. Note that during both the 5.5 as well as 2.5 hours there the download of additional packages was finished by then.
 Eventually I saw the relieving dialog box on the screen: 


 +-------------------+ [!!] Finish the installation +--------------------+
 |                         Installation complete                         |
 | Installation is complete, so it is time to boot into your new system. |
 | Make sure to remove the installation media (CD-ROM, floppies), so     |
 | that you boot into the new system rather than restarting the          |
 | installation.                                                         |
 |                                                                       |
 |     <Go Back>                                          <Continue>     |
 |                                                                       |
 +-----------------------------------------------------------------------+

Great. I continued, and later on I called it a day. Next day however Dapper Drake didn't want to boot: 

{0} ok boot

SC Alert: Host System has Reset
0ALERT! /dev/sda5 does not exist. Dropping to a shell!


BusyBox v1.01 (Debian 1:1.01-4ubuntu3) Built-in shell (ash)
Enter 'help' for a list of built-in commands.

/bin/sh: can't access tty; job control turned off
#

A closer look revealed that the XFS module wasn't loaded. cat /proc/filesystems was telling me too that XFS wasn't supported. The culprit was probably the installer again which failed to include xfs.ko in the initrd. At this point of time I rather cut my losses in terms of time, somewhat unsatisfied due to the amount of it spent already for this project.

Conclusion

The T2000 hardware made a good and solid impression as it is almost usual from Sun. The Niagara's are not bargains as opposed to e.g. cheap Intel/AMD-based hardware and they are not general purpose machines for every data center application, e.g. Niagaras are not targetting HPC floating point applications, since the open sourced T1 has only one FPU. The strength besides the low energy consumption is the multiprocessing/-threading technology. The good thing is that you (still) can try before you buy, that should help you to make a purchase decision without risk.
 A minor point I ran into is that the speed over the Ethernet console as well as over the serial console seems to be 9600 bps only.

As far as the software side is concerned: Solaris 10 U2 – as expected – installed without any problems on the T2000 and the new features especially the Zettabyte file system brought Solaris another step forward. As far as Ubuntu is concerned: Except a few glitches I am using the x86-version of Dapper Drake as a good Debian-based Linux distribution since its release. However this impression is opposed to the UltraSPARC edition: Besides some dents and glitches non-existing in the Intel-world of Ubuntu, the installer alone has a number of real problems (see Summary). Politely speaking: Under this circumstances it's hard to understand why Ubuntu claims the T2000 be certified hardware for their Dapper Drake operating system. Later on Sun told me that the SAS controller was changed in newer revisions of the T2000 hardware which the next release Edgy Eft addresses. Besides that there's still some amount of homework which the Ubuntu team needs to get done. Disregarding for a moment the usability of the installer over the serial line: customers want their data to be safe, the installation should not take an eternity. Also the Sun Blade 1000 installation didn't succeed, see right hand side.

Not that I don't like Linux – part of my profession is providing services for Linux, that I've been doing since more than a decade – but the experience I've made during this research confirms my in the beginning of this review mentioned impression of the past: "What's the point running Linux on a SPARC, there's a better OS".
  As much as I appreciate personally Sun Microsystem's embracement of (currently two) Linux flavors. More consequent would be if in addition they would give support in terms of manpower to the Linux/T1 projects. This also should include establishing a communication process to the Linux maintainers if the underlying hardware changed, so that they can address and also document them. Another idea would be to do some in-house testing within Sun whether the flavor of Linux they embrace on their platforms is good enough to promote as an alternative to their own operating system for business.

Dirk Wetter, 7/10/2006

Discuss this article. « Back | del.icio.us

Discussions

Ivan Krstić (member of the Ubuntu server team) wrote (10/16/2006, 03:18 PM):

In revision2 of the T2000 hardware, the SAS controller moved on-board (it used to be on a PCI card), and the current Dapper kernel lacks support for the on-board controller. The kernel in Edgy (to be released close to the end of October) supports it, however, so you can grab a daily build for testing: http://cdimage.ubuntu.com/ubuntu-server/daily/current/. We're going to see if we can backport the on-board SAS controller support to the Dapper kernel.

Permalink, Comments [0], Reply


Jonathan wrote (10/20/2006, 01:54 AM):

I have been keeping an eye on your "Cool threads and hot nerves" article, because it seems to be the closest thing to the situation I have been working through. I ended up giving up on the Ubuntu 6.06.x install, it was a bit too messed up of a distribution for me to work with, so many things not working that should have if it were to have been put together correctly. I just needed to get a working Linux environment up and running so I could start development on porting openvz to sparc64, so what I did was install "Ubuntu 6.10" from CD, and even though it is in beta, it is by far more stable and a more complete distribution than the 6.06 line. The install went perfect and everything is up and running just as it should. So depending on your goal, that may be an option you might want to consider. Also, I do have some notes on my 6.06 install attempts that may help.

Permalink, Comments [0], Reply


Jonathan wrote (10/20/2006, 10:31 AM):

I agree with your point of view about how the so called "certified" for the T2000 Dapper Drake (net install, or otherwise) does not seem to even be able to install correctly on the Sun Fire T2000 (I am using the 6 core, 8 gigs of ram, and 2 SAS drive version). Coming in to the situation with very little experience with Ubuntu, with this experience I must say, it does not leave a good taste in my mouth, and if that is not the normal quality of their distributions, then yes, embarrassment indeed. And Sun's endorsement, I am still scratching my head at the whole situation.
 The system I am using is from the 60 day trial deal as well. I jumped at the chance to help make such smart hardware design (except for the 9600 bps thing, and the limited SCSI subsystem) come into the Linux community. I do allot of work with Virtuozzo and OpenVZ, and the extreme multicore/multithread design of the T1 CPU has great potential in these virtualization systems. Plus I am thrilled at the low power consumption and heat generation of the system. All in all, it looks like the bottleneck for this system is going to be the SCSI subsystem, which in the future, I am sure will be remedied, so I am still interested in enhancing the Linux support of this CPU architecture. Too much energy is blown out the wall of a datacenter these days, and not only does that directly translate to cost, but it costs more and more to fit the datacenter with the equipment needed to move that heat out. I could go on and on about how improved efficiency is needed, but that is a different project.

Permalink, Comments [0], Reply


Ivan Krstić (member of the Ubuntu server team) wrote (10/23/2006, 03:18 PM):

We actually looked into backporting support for rev2 into the Dapper kernel, but found it's not feasible due to invasive changes in the underlying subsystem code. Edgy works with rev2 out of the box, however (I have it running with /boot on RAID, and root on LVM on RAID.)

Permalink, Comments [0], Reply


Dirk Wetter wrote (10/25/2006, 22:40 PM):

Addendum: Fabio (Massimo Di Nitto, the head of the Ubuntu server team) confirmed the ZFS bug I found, however was leaking through that's not going to be fixed in the upcoming Edgy Eft. Worse: "We can't look at the [ZFS] code because of the licence or there might be contamination issues. GPL is not compatible with CDDL." Great, that means that nobody is going to fix this though Sun put the ZFS source code a mouse click away? I was wondering about two things:

  1. This must be an issue on a dual boot AMD/Intel machines (Solaris x86/Linux), too! Almighty Google however couldn't give me a pointer.
  2. There's the ZFS on FUSE project: If a Linux developer is afraid of getting of spoiled just by looking at the CDDL code (SCNR), how about just asking Ricardo Correia, the developer of ZFS on FUSE, how he detects the pools or any other person involved in ZFS development?

I respect the GPL, but I also appreciated that Sun open sourced their code. Here I think it's not a license issue and dogmatism should make place for pragmatism. Is endagering data not an argument? Besides: both worlds can learn from each other, technically speaking it's a pity if there's a big gap dividing both.

Permalink, Comments [2], Reply

   Detlef Drewanz (Technical Specialist and OS Ambassador, Sun Germany) wrote (10/30/2006, 1:40 PM):

For Solaris x86 starting from Solaris 10 1/06, the fdisk(1M) identifier is set to 0xbf to facilitate the configuration of multiboot environments that include the Linux operating system. In previous Solaris releases, the Solaris fdisk(1M) identifier was set to 0x82, the same identifier as the Linux swap partition.

Comments [1], Reply
  
Dirk Wetter wrote (10/30/2006, 7:40 PM):

Thanks for the useful hint! So any flavor of Linux should friendly coexist on x86 with any Solaris partitions, not only the slices formatted with ZFS.

Comments [0], Reply


Dirk Wetter wrote (11/6/2006, 09:01 PM):

John's performance is actually due to the pipeline design of the T1 processor as Thomas Nau (University of Ulm) outlined on a conference in Singapur: this year: "single in-order issue pipeline of T1 versus 4-way superscalar one of the IIe processor". According to his research consistent coloring makes jobs scale better for >=12 threads.

Permalink, Comments [0], Reply


Steven Noels (Developer of Open Source CMS Daisy) wrote (11/21/2006, 10:19 AM):

I saw your great write-up about your installation of Ubuntu on a T2000, and was wondering whether you would deem Ubuntu/T2000 a worthy environment for some Java-centric development and hosting work. Does Java run well on the T2000 under Ubuntu? Does Ubuntu run well on the T2000?

Permalink, Comments [3], Reply

   Dirk Wetter wrote (11/21/2006, 3:53 PM):

You hit the nail on the head, actually I should rather say "nails" since it's a two edged matter:

  • As I stated the CD image of LTS version Dapper Drake doesn't run on the new revisions on the hardware. The netboot image is somewhat a pain to install (console + installation speed). Edgy Eft should be fine, however that doesn't come with (L)ong (T)erm (S)upport.
  • Nail number 2: The only JVM I know about which runs on Linux/SPARC is the one from blackdown.org: the last version available for Linux/SPARC is 1.2 which nobody wants anymore. 1.5 is for all architectures, including Linux/SPARC, in the works. IBM probably doesn't have a strong interest for supporting that platform either w/ their version of Java. Maybe things change once Sun's announcement GPL'ing Java becomes effective and somebody ports the Sun JVM to Linux/SPARC.

Comments [2], Reply

   Steven Noels (Developer of Open Source CMS Daisy) wrote (11/21/2006, 09:06 PM):

Ouch. Having no Sun JDK for Linux on Sparc64 means an end to my explorations, then. We're a Java-centric company, and robust Java 1.5 VMs is the least we expect from a decent server environment. I went looking hopefully at https://launchpad.net/distros/ubuntu/+source/sun-java5 however I reckon 'sparc' doesn't mean Niagara support, yet. :/

Comments [1], Reply

   Dirk Wetter wrote (11/21/2006, 9:15 PM):

Thanks for the hint: At https://launchpad.net/distros/ubuntu/edgy/sparc/sun-java5-jre/1.5.0-08-0ubuntu1 there's the JRE at least, but I couldn't find a JDK @ https://launchpad.net/distros/ubuntu/edgy/+source/sun-java5.

Comments [0], Reply

Discuss this article. « Back | del.icio.us

Summary

Dapper Drake Linux on the T2000 tested has a number of problems:
  • Both the 6.06 and 6.06.1 cd images didn't work because they seem to lack support for the (new) SAS controller.
  • Due to the installation method it's not possible to debug the problem (lsmod, dmesg) or find a workaround.
The Dapper Drake netboot image works, however:
  • The package installation took very long (>9h)
  • This image doesn't contain a SMP kernel
  • The partitioner wipes out ZFS pools without warning, thinking it is free disk space
  • After installation Ubuntu couldn't find the root partition. The suspect is the lack of XFS support in the initial ramdisk.

Ubuntu 6.06.1 on a Sun Blade 1000

For two reasons Dapper Drake's installation was also tested on a spare Sun Blade 1000 (Ultra III CPU, internal FC disks):

  • to test how good Dapper Drake is doing on a non-highend platform
  • practical reasons: to get sooner used to the installation procedure which is faster on the Sun Blade

Prerequisites: [..to be completed soon, please return in a few days..]

 
Footnotes

1) During mid October this link was broken. After making sure that all your equipment – the switch and the NICs where you plan to talk to – supports jumbo frames in general, pick a frame size which is supported by every port. Continue to follow the instructions(Google cache). Note that jumbo frames on the T1000 currently seem to lack support.

2) The image was pulled via wget from the server which doesn't touch the mtime, md5 is 5b0290d9f927c9ed08d0ee4e3cb93ad9

3) According to the research done on the Sun Blade 1000 the Dapper Drake installer seems to lack support for speeds other than 9600 bps. Also SILO was not configured to make use of the serial line.