My Wushu Blog

I need to share this. When I google for “Samba performance”, I never see real numbers, real configuration files, or real hardware environments. All I read are anecdotal recollections, and that is not good enough. I like numbers, and I’ll let the numbers speak for themselves:

    > netstat -I em0 -w 1
                input          (em0)           output
       packets  errs      bytes    packets  errs      bytes colls
         90166     0   98762637      95363     0    5332847     0
         18131     0   24713156      20042     0    1123684     0
             4     0        310          1     0        178     0
             8     0        518          1     0        178     0
         10153     0   10952920      10696     0     598129     0
         92990     0  102837002      98476     0    5514994     0
         92025     0  102680574      97277     0    5439496     0
         92080     0  101799874      97403     0    5448637     0
         75348     0   90861608      80972     0    4537737     0
         90895     0  100323946      95781     0    5360948     0
         89313     0   97371154      94364     0    5278618     0
         81363     0   89229738      85861     0    4803589     0
             2     0        126          3     0        286     0

I was so shocked that I had to use gstat and zpool iostat to verify the information:

    dT: 1.002s  w: 1.000s  filter: da0
     L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w   %busy Name
       35   1476      0      0    0.0   1476 188421   23.7  100.0| da0

    > zpool iostat  1
                   capacity     operations    bandwidth
    pool         used  avail   read  write   read  write
    ----------  -----  -----  -----  -----  -----  -----
    tank        5.68T  4.32T      1     81   250K  10.1M
    tank        5.68T  4.32T      0  1.37K      0   175M
    tank        5.68T  4.32T      0  1.44K      0   184M
    tank        5.68T  4.32T      0  1.44K      0   184M
    tank        5.68T  4.32T      0  1.44K      0   184M
    tank        5.68T  4.32T      0  1.44K      0   184M
    tank        5.68T  4.32T      0  1.44K      0   184M
    tank        5.68T  4.32T      0  1.44K      0   184M

This is all through Samba (3.3.9), There was no local work being done. I unfortunately didn’t configure MRTG correctly, so it had built a malformed graph while all this happened. Having a picture from all of this would have been nice.

The underlying storage is a SATABoy2 RAID6 array, with a simple “flat” ZFS filesystem (version 13). As cheap as the SATABoy’s are (and come on, they have a terrible IIS web interface), they can at least keep up with the current load.

I have felt that if you are going to use ZFS, you should let it manage the RAID, and not bother with a hardware RAID controller. While the hardware RAID may be faster, ZFS’s ability to self-correct bad blocks is a great feature despite the performance set back. However, RAID6 is pretty good in itself, and having dual parity would ideally reduce the risk of a bad block being detrimental.

One thing I noticed with Samba is it doesn’t seem to be a threaded daemon. When I do a top(1) -H, there are only 2-3 smbd processes, and one of them is running around 30%. Though I don’t really know how well Samba can scale out, this environment only has about 10 users. I would like to see how samba reacts if there are a couple hundred active users. Furthermore, how does a native Windows server handle a couple hundred users? It may handle it a little better, however, I don’t think I would enjoy watching NTFS handling a multi-terabyte volume… it would be like watching a stroke victim eat a bowl of soup. I do admit I am biased and I have no working experience with Windows as a large file server, most of them that I have worked on are horribly limited and underpowered, and no one seems to care if they perform well or not.

Hardware

CPU information

    Machine class:    amd64
    CPU Model:    Dual Core AMD Opteron(tm) Processor 285
    No. of Cores:    4
    Cores per CPU:

RAM information

    Memory information from dmidecode(8)
    Maximum Capacity: 8 GB
    Number Of Devices: 4
    Maximum Capacity: 8 GB
    Number Of Devices: 4

    INFO: Run `dmidecode -t memory` to see further information.

    System memory summary
    Total real memory available:    8048 MB
    Logically used memory:        2876 MB
    Logically available memory:    5172 MB

    Swap information
    Device          1K-blocks     Used    Avail Capacity
    /dev/da1s1b       8373844      28K     8.0G     0%

Storage information

    Available hard drives:
    cd0:  Removable CD-ROM SCSI-0 device
    cd0: 1.000MB/s transfers
    da2:  Fixed Direct Access SCSI-5 device
    da2: 300.000MB/s transfers
    da2: Command Queueing enabled
    da2: 140009MB (286739329 512 byte sectors: 255H 63S/T 17848C)
    da1:  Fixed Direct Access SCSI-2 device
    da1: 300.000MB/s transfers
    da1: Command Queueing enabled
    da1: 69618MB (142577664 512 byte sectors: 255H 63S/T 8875C)
    da0:  Fixed Direct Access SCSI-5 device
    da0: 200.000MB/s transfers
    da0: Command Queueing enabled
    da0: 10491861MB (21487333120 512 byte sectors: 255H 63S/T 1337524C)

    Raid controllers:
    umass-sim0:
    mpt0:
    vendor='LSI Logic (Was: Symbios Logic, NCR)'
    device='SAS 3000 series, 4-port with 1064 -StorPort'
    isp0:
    vendor='QLogic Corporation'
    device='QLA6322 Fibre Channel Adapter'

    Currently mounted filesystems:
    /dev/da1s1a on /
    devfs on /dev
    tank on /tank
    /dev/ufs/EXPORT on /export

    I/O statistics:
           tty             da0              da1              da2             cpu
     tin  tout  KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s  us ni sy in id
       0    40 63.61 167 10.36  16.53   2  0.03  61.65   0  0.00   1  0  4  0 94
    INFO: Run iostat(8) or gstat(8) to see live statistics.

    Disk usage:
    Filesystem         Size    Used   Avail Capacity  Mounted on
    /dev/da1s1a         58G    3.4G     50G     6%    /
    devfs              1.0K    1.0K      0B   100%    /dev
    tank               9.8T    5.7T    4.1T    58%    /tank
    /dev/ufs/EXPORT    126G    148K    116G     0%    /export

Software

• FreeBSD 8.0-RELEASE-p1 FreeBSD 8.0-RELEASE-p1 amd64
• samba-3.3.9 A free SMB and CIFS client and server for UNIX

Samba 3.3.9 Compile-Time Config

> make showconfig
===> The following configuration options are available for samba-3.3.9:
     LDAP=on "With LDAP support"
     ADS=on "With Active Directory support"
     CUPS=off "With CUPS printing support"
     WINBIND=on "With WinBIND support"
     SWAT=off "With SWAT WebGUI"
     ACL_SUPPORT=on "With ACL support"
     AIO_SUPPORT=on "With Asyncronous IO support"
     FAM_SUPPORT=on "With File Alteration Monitor"
     SYSLOG=on "With Syslog support"
     QUOTAS=on "With Disk quota support"
     UTMP=off "With UTMP accounting support"
     PAM_SMBPASS=on "With PAM authentication vs passdb backends"
     DNSUPDATE=off "With dynamic DNS update(require ADS)"
     DNSSD=off "With DNS service discovery support"
     EXP_MODULES=on "With experimental modules"
     POPT=on "With system-wide POPT library"
     MAX_DEBUG=off "With maximum debugging"
     SMBTORTURE=off "With smbtorture"
===> Use 'make config' to modify these settings

System Tuning

The Kernel

I enabled device polling, and took out debugging in the kernel (Sanders, get it! Mmm, I’m hungry…)

diff /usr/src/sys/amd64/conf/GENERIC /usr/src/sys/amd64/conf/SANDERS
    33d32
    < makeoptions    DEBUG=-g        # Build kernel with gdb(1) debug symbols
    78c77
    <
    ---
    > options        DEVICE_POLLING

/boot/loader.conf

    ispfw_load="YES"
    kern.hz="2000"
    aio_load="YES"

/etc/sysctl.conf

    kern.coredump=0
    security.bsd.see_other_uids=0
    security.bsd.see_other_gids=0
    kern.ipc.maxsockbuf=16777216
    kern.ipc.nmbclusters=32768
    kern.ipc.somaxconn=32768
    kern.maxfiles=65536
    kern.maxfilesperproc=32768
    kern.maxvnodes=800000
    net.inet.tcp.delayed_ack=0
    net.inet.tcp.inflight.enable=0
    net.inet.tcp.path_mtu_discovery=0
    net.inet.tcp.recvbuf_auto=1
    net.inet.tcp.recvbuf_inc=524288
    net.inet.tcp.recvbuf_max=16777216
    net.inet.tcp.recvspace=65536
    net.inet.tcp.rfc1323=1
    net.inet.tcp.sendbuf_auto=1
    net.inet.tcp.sendbuf_inc=524288
    net.inet.tcp.sendspace=65536
    net.inet.udp.maxdgram=57344
    net.inet.udp.recvspace=65536
    net.local.stream.recvspace=65536
    net.inet.tcp.sendbuf_max=16777216
    net.inet.tcp.mssdflt=1460

rc.conf (em0 flags)

    ifconfig_em0="inet xxx.xxx.xxx.xxx  netmask 255.255.255.0 polling tso mtu 9194"

smb.conf

        min receivefile size = 131072
        aio read size = 1
        aio write size = 1
        use sendfile = yes
        lock directory = /var/run/samba/
        keepalive = 300

I’m also using LDAP users and group. I wasn’t sure if there would be a noticible performance hit for local users or LDAP users. There doesn’t seem to be one.

We use Active Directory, and since Quest/Vintela still won’t make a FreeBSD client for the Quest Authentication Servers ( a sales rep once told me “There are just too many versions of BSD…”) , I have to use all the open source utilities like OpenSSL, OpenLDAP Client and Kerberos. I don’t mind having to do it, but it is always nice if you can maintain one standard process across ALL systems, and we have a lot more Linux and Solaris systems than FreeBSD. I’m the odd one.

That aside, I use the latest OpenSSL in FreeBSD 8.0, OpenLDAP 2.4.20, and the built-in version of Heimdal Kerberos.

I get similar performance form NFS, however, most desktop users have are either on a Windows or OS X, and CIFS seems to be the unifying network storage protocol.

One thing I have yet to really figure out is configuring Samba to use proper NT ACL’s. However, if you can live with UNIX style permissions, a setup like this is pretty good at serving out lots and lots of data. Maybe that will be next.

mike Geekyness FreeBSD, kickassery, Samba, ZFS

I recently had a little problem with a new FreeBSD install, and it is one of those times were I sort of appreciate how FreeBSD assigns device handles, yet at the same time hate it

The setup is this:
The OS was installed on a mirrored hardware raid device (using the mpt(4) driver), and then I had a large RAID6 array attached via a FC controller (using the isp(4) driver). When I installed the OS, the mpt device was showing up as da0. So I went ahead with the install and rebooted the system, so far so good.

What I didn’t realize was the FC device was not seen yet, so after some fiddling, Jenny and I got the large RAID6 array to show up… unfortunately, the isp card was before the mpt card on the PCI bus:

isp0@pci0:2:1:0: class=0x0c0400 card=0x01321077 chip=0x63221077 rev=0x03 hdr=0x00
vendor = 'QLogic Corporation'
device = 'QLA6322 Fibre Channel Adapter'
class = serial bus
subclass = Fibre Channel
mpt0@pci0:2:3:0: class=0x010000 card=0x30601000 chip=0x00501000 rev=0x02 hdr=0x00
vendor = 'LSI Logic (Was: Symbios Logic, NCR)'
device = 'SAS 3000 series, 4-port with 1064 -StorPort'
class = mass storage
subclass = SCSI

and the RAID6 now became da0, and the OS device now became da1.

Doh!

The system prompted for the / drive, so I had to call out the correct device at the mount> prompt:

mount> ufs:/dev/da1s1a

After that, the system continue to boot into mult-user mode, which cause some very strange console behavior (it acted like the return key was being held down), and my only option was to SSH in as local user, su to root, and then fix /etc/fstab.

This was not devastating, however, it show the importance of using disk labels instead of device handles in certain use cases. I haven’t fixed the / mount, but to get a comfort level with using GEOM labels I added another drive to the system and called it EXPORT.

You can assign a permanent label in two ways (that I know of). When you newfs the device, you can specify the L flag (BTW, -O2 means to use UFS2, and -U will use Soft-Updates):
[root@paper ~]> newfs -O2 -U -L EXPORT /dev/da2s1a
OR using glabel (which is what you would have to do for a non UFS filesystem.
[root@paper ~]> glabel create EXPORT da2s1a
Now we can see our newly labeled device in action:
[root@paper ~]> ls /dev/label
. .. EXPORT
[root@paper ~]> glabel status
Name Status Components
label/EXPORT N/A da2s1a

To add it to /etc/fstab, you can either edit the file, or append the correctly tab-delimited line like so:

[root@paper ~]> echo "/dev/label/EXPORT\t/export\tufs\trw\t2\t2" >> /etc/fstab
[root@paper ~]> mkdir /export
[root@paper ~]> mount export

Hurray!

[root@paper ~]> df
Filesystem 1K-blocks Used Avail Capacity Mounted on
/dev/da1s1a 60931274 4754540 51302234 8% /
devfs 1 1 0 100% /dev
tank 10569645824 107237376 10462408448 1% /tank
/dev/label/EXPORT 132022788 4 121460962 0% /export

[root@paper ~]> mount
/dev/da1s1a on / (ufs, local, soft-updates)
devfs on /dev (devfs, local, multilabel)
tank on /tank (zfs, NFS exported, local)
/dev/label/EXPORT on /export (ufs, local)


This is now a persistent label. To be safe, I’ll have to boot off of a CD/USB drive and modify the root device.

mike Geekyness FreeBSD, GEOM, Jenny, Labels

I’ve always had a vested interest in the entire nvidia display driver for FreeBSD project, and I’ve pretty attached to the project. So much, that back in 2001 I started a little petition, got enough attention (and more importantly, a large list of people who signed my petition), and ever since 2002 FreeBSD users have been able to use high quality nvidia drivers. It wasn’t all me, whoever ran nvidia.netexplorer.org asked me to combine efforts, and I gave them my list, and they continued to market it and work with some folks at nvidia.

It is really nice to see that both the FreeBSD team and nvidia have worked together to do the necessary kernel development and get a 64bit driver. I used to use FreeBSD as my primary desktop at work, and it was great to use the hardware drivers for my displays. What is also nice is people in the nvidia forums are also asking for CUDA drivers on FreeBSD, that would be slick as well.

Digg the story if you want too:
http://digg.com/linux_unix/Official_64bit_NVIDIA_drivers_for_FreeBSD

mike Geekyness FreeBSD, nvidia

So, it looks like FreeBSD 8.0 has been pre-released; the official date is going to be 11/25, as noted in src/UPDATING:

Updating Information for FreeBSD current users

This file is maintained and copyrighted by M. Warner Losh
.  See end of file for further details.  For commonly
done items, please see the COMMON ITEMS: section later in the file.

Items affecting the ports and packages system can be found in
/usr/ports/UPDATING.  Please read that file before running
portupgrade.

NOTE TO PEOPLE WHO THINK THAT FreeBSD 8.x IS SLOW ON IA64 OR SUN4V:
        For ia64 the INVARIANTS and INVARIANT_SUPPORT kernel options
        were left in the GENERIC kernel because the kernel does not
        work properly without them.  For sun4v all of the normal kernel
        debugging tools present in HEAD were left in place because
        sun4v support still needs work to become production ready.

20091125:
        8.0-RELEASE.
...

Thanks for the warning, and I don’t feel that 8.0 is slow in any way

You can now update to FreeBSD 8.0 with either syncing your source with csup:

*default host=cvsup.FreeBSD.org
*default base=/usr
*default prefix=/usr
*default delete use-rel-suffix
*default compress
src-all release=cvs tag=RELENG_8_0

Or with freebsd-update(8):

# freebsd-update -r 8.0-RELEASE upgrade

then

# freebsd-update install

and after the reboot, possibly another round of ‘freebsd-update install” to finish things up. You can actually upgrade from 7.2 to 8.0, which is pretty impressive since they are considered major releases (and minor release upgrades work just fine as well).

Why would you upgrade to 8.0 over 7.2? Well, Ivan Voras already has a very nice page on the notable features in 8:
http://ivoras.sharanet.org/freebsd/freebsd8.html
In case you want my short list version of that, here are the big highlights for me:

• Kernel Stuff
  • Kernel limit on amd64 increased (this greatly benefits ZFS)
  • Superpages
  • Network stack virtualization, equal cost multipath routing and other really cool network improvements
  • NGROUPS has been increased from 16 to 1024
  • Other kernel improvements like light weight threads, the new ULE 3.0 Scheduler
  • NFS Locking
  • Qlogic 8GB HBA support
  • New AHCI driver
• Userland Stuff
  • Parallel port builds
  • Jails v2
  • Dtrace
  • CLANG/LLVM Compiler

One of the cool things about FreeBSD is its focus on improving what is there. There have been some really big additions to FreeBSD from time to time, but overall, the goal has been to constantly refine and improve the performance. That is what I’m mostly excited about, the continual refinement of an already robust OS.

There are other features, like CLANG and LLVM or Dtrace, where I’m excited about them, but only because I can’t wait to see how others use them. I myself cannot obtain a lot of useful information from Dtrace, however, a kernel developer who knows what they are doing probably can, and that helps them out (which sometimes helps me out).

I’ve used the BETA and RC versions of 8.0, so not only was I pleased with the experience, I’m also excited to see its adoption with the new improvements. I’ve seen some PostgreSQL and MySQL benchmarks and there was a clear performance gain between 7 and 8.

Now is also a good time to mention that the FreeBSD Foundation is rounding up this years donations.

It’s pretty amazing that FreeBSD is a non-profit group; they do not have a CEO, a marketing department, or a horde of full-time developers… and yet they put out a extremely well engineered OS ( that is the boon of not having a marketing department all decisions are driven by the community demand and the developers, and not buzz-words like “the cloud”) with a killer network stack, and over 22,000 available ports.

mike Geekyness FreeBSD

Step 1: The Question

Background:
I was watching a new episode of Good Eats (Season 13, Episode 7) and Alton Brown was going over a good coffee recipe he called “Man Coffee”. It had a decent ratio of grounds to water, a coarse grind, and he employed the use of a French Press. Then, to my surprise, he added a bit of Kosher Salt.

So, being the unapologetic AB fan that I am, I decided to see how the addition of salt would taste. I had it on my coffee over the last two weeks, off and on, and I decided that I liked the additional flavor. Michele was skeptical, and didn’t think I could actually tell the difference and it was all in my fan-boy head. I admitted this was a likely scenario, and as a good scientist I have to be honest with the data; and that is the Placebo Effect does have a measurable outcome. So, I told her that I would do a blind taste test and see if there was any noticeable flavor in salted coffee.

So, the question is: Can I taste the difference between regular coffee, and coffee brewed with kosher salt.

Step 2: Research

I did a little google-ing about why people put salt in their coffee. The consensus seemed to be that it can help take away the bitterness, however, if you use enough coffee grounds you should not taste any of the acidic bitterness. Coffee becomes bitter for two reasons:

• the essential oils are washed away during the extraction process, leaving the acidic compounds (those coffee compounds are volatile, so you also have to grind the beans and brew them quickly!)
• the coffee is left on a burner longer than it should

Okay, there are more than two, but I’m not a barista or a chemist so that is all I can cover

I normally grind and brew immediately, and I use about 6 tablespoons for 4 cups of coffee which is “a lot”. I think I avoid the common bitterness pitfalls, and adding salt (to me) just seemed to add more body to the coffee.

The test is only to answer the question about detection, and not if it enhances the coffee’s flavor. That would have to be another trial, and I would need more voulenteers and cups…

Step 3: Propose a Hypothesis

I hypothesize that a pinch of salt to the coffee grounds does add a noticeable effect.

That is pretty straight-forward

Step 4: Test with an Experiment

I decided on a regular blind taste test. I wanted to do a double blind test, however, since I was the subject and Caralyne can’t make coffee, Michele had to brew and pour the samples. I also had to settle for 4 cups (those were the only clean and similar cups we had), I wanted a dozen at least so I would have a large enough sample size.

The problem with this test is that both Michele and I are biased; she knew which cup had what, and I knew what the test was about and what I was looking for in the coffee.

The test is simple, I ground up about 12 tablespoon’s worth of Kirkland Coffee.

I put 6 in a cup with a pinch of salt, and 6 all alone. Each one was brewed with the same coffee maker, with 4 “cups” of water and poured into identical plastic pitchers.

Michele labeled the tasting cups, and I sat down to have a taste of each one, marking which ones I liked more than the others.

Step 5: Analyze Results, Draw a Conclusion

After my 4 samples, Michele and I both decided that there was a difference in taste, and I did prefer the one with salt. She didn’t like either, but that is normal since she doesn’t like the strong coffee I make anyway (she calls it “Cat Pee coffee”, which makes me think of Tiger Brand Coffee:

Tiger Brand Coffee is a real treat, even tigers prefer a cup of it to real meat

).

Conclusion

The salt does add flavor, so this is almost like me drinking a Holiday Spice or Hazelnut coffee. It’s not real untampered coffee, but it is a nice treat. I also tend to drink more water in the morning when I have it, so thats not a bad thing since coffee is a diuretic.

I’d like to acknowledge

• Alton Brown and his excellent show Good Eats
• Science Buddies dot org for their excellent page on The Scientific Method
• Wikipedia for their articles on Blind and Double Blind Testing
• My good friend and co-worker, Jenny, for giving me some great ideas on how to help narrow the test down to get useful results.
• Mr Swarm, my 7th grade science teacher at Park Jr. High, who introduced an entire class to the Scientific Method and let us perform real experiments with fire and food.
• The Skeptics Guide to the Universe, a great science podcast that teaches me the difference between “theory” and “hypothesis”

Have a Scien-tastic day!

mike Geekyness blind taste test, coffee, Good Eats, the scientific method

I've been a "PC" user since I was in grade school. Well, technically, the first computer I owned was a hand-me-down Atari ST that had a broken printer and a flight simulator. That almost makes me a "Mac" user since it was based on the same hardware (Motorola 68K cpu). All of my (3) friends had computers, all PC's running DOS, and their primary motivation was to play all of the cool adventure games from Sierra Entertainment. So, when I had a very unsuccessful time getting Conquests of Camelot to run on my Atari ST I was pretty motivated to join my geeky buddies. Oh the months of begging and pleading that followed, fun times indeed. Nostalgia aside, I've been using a computer primarily for games (only around 98 did I start running other OS's) for a large portion of my life. I started with MS-DOS 5.0 on a Packard Bell 386SX 20Mhz with 1MB shared memory: All the while, I've never actually paid for an operating system. I bought a copy of SuSE Linux once, and a copy of FreeBSD at CompUSA years ago. Those were only $10 - $20 at most. Commercial OS's have always been pretty pricey from my point of view, and they have only gotten more expensive over the years. What typically happens is I find a way to get the OS for free:

• Vista - Free, I took a survey and ran some monitoring software for 3 months
• XP - Free, I got an academic license from DVC
• Windows 2000 Pro - Free, I got a copy when I worked at Great Entertaining.com. So I could "work from home"...
• Windows 98SE - "Free"... a friend of a friend brought a few copies back from Taiwan.

Every OS before that was either OEM (like from a Packard Bell), or a friend let me copy it... there, I admit it. I used my friends copy of MS-DOS 6.20, Windows 3.11 as well. To be fair, it was so un-usable on my 486 that I had to remove it. It also took up a large portion of my 100MB Dae Woo hard drive. So what has changed? Why did I actually BUY a copy??? It is not because of features or my love for Microsoft. It is for three reasons:

• I have a copy of Vista Ultimate (64bit) that I got for FREE by taking a survey and running software for 3 months.
• I have a Directx10 GFX card and a decent catalog of DX10 games now.
• Getting a free and legitimate copy has become very difficult now.

With that, I was able to purchase the Upgrade edition which revoked my Vista product ID. I never used Vista because I disliked it, and it wouldn't boot after its first round of updates. The install of Windows 7 was much nicer, and a lot quicker. I do enjoy its substantially faster boot times, even compared to XP, and it is really nice not to have to fetch drivers for all my hardware. Even the all-in-one canon printer we just got, it worked right off the bat which is normally unheard of in the Windows world. There are a few things I wish Windows would do; mostly things that I enjoy from other coolers OS's like FreeBSD and Linux:

• Virtual CD/DVD devices - I still have to use a 3rd party tool to mount ISO's
• Mount Points - I HATE referring to device handles like 'C:' and 'D:', it reminds me of VMS
• Support for the mouse wheel in everything. The Gnome environment excels at this. It is such a simple concept that I'm really amazed other OS's don't do it
• Built-in SSH client - This is knit-picking, but it would be awesome to have
• Cool GNU tools like md5sum, gunzip, tar, lynx, etc...

I can still accomplish all of the above with toolsets like gnuwin32, putty, etc.. it would have been nice not to rely on that. Nice trip down memory lane, and how about that timeline huh? That took me a while to figure out, it is called SIMILE Timeline, and they had a nice example in their documentation. It's fun to find new ways to represent information. [inline] [/inline]

mike Geekyness Packard Bell, timeline, Windows 7

This was a very cool conference. I picked up a lot of useful information on both the open source tool, Puppet, and some ideas on infrastructure. What also made this conference unique, is how honest the Puppet team and community were about the projects strength and weaknesses. Those that have deployed Puppet on a larger scale (MessageOne and Google) seemed to go through the same iterations in attempting to scale out their Puppetmaster's. From WEBrick (which is what I'm currently running Puppet with ), which is hated by all since its a single process/thread web server that can only handle one request at a time. To Mongrel, which you have to manage a mongrel cluster script, feed it lots of memory, and then throw an apache proxy server in front of them. Now, people are starting to settle on using Passenger/mod_rack, which is what I spent most of yesterday looking into and setting up. This allows apache to mount a rails instance, and then you don't actually have to run puppetmasterd. This still requires some decent hardware, and I'm currently running my puppetmaster on a VM with 2GB or memory, so I'll have to watch out for that. Chris, the one who introduced me to Puppet, said he still uses WEBrick for all of his DB, Tomcat, and Apache servers (I think he said something like 200 systems) and it has been working out nicely. He, like the guys at Google, also doesn't run puppet as a daemon. Anyway, the point is, we learned a lot about the project, way more than if a sales person had come to us and just told us the things puppet does well, or how it operates on paper (*cough* LANDesk *cough*). It was really awesome to talk with Andrew Pollock and Nigel Kersten from Google. See, I was a little unsure about Puppet in our environment, where we have multi-purpose servers, computer servers, and desktops that we have to manage. It seemed, at a first glance, that most of the Puppet users out there have a homogeneous environment, and Andrew (Shafer) had stressed the concept of single role servers. After talking with them, I felt a lot more comfortable pursuing Puppet across our servers and desktops. Did I mention they were super cool and friendly? We also learned a lot about the Puppet developers, which had its own interesting advantage. I have a lot of respect for what Luke Kanies has been able to do, and by the end of the conference, he showed significant mastery in what he has done, as well as some humility and admitting what he has not been able to do and why. I was a little put off the first day though, when both him and Andrew came off a little arrogant and crass. It did make me step back and think, "Is this project going to be well managed in the future with personalities like this in charge? Is their answer of 'don't do that!' tongue in cheek, or are they not supportive of a diverse environment?". In the end, I have more respect for the project than ever, and with it still being a young project, I hope they listened to some of the feedback, and I also can't wait to see where it ends up in the next year. Andrew, the Puppet Andrew, came up to us a lot during the conference, and he was fun to talk too, and he's very academic and he had a lot of abstract concepts to talk about. Also, he said this was the first conference he has arranged, and I think he did a fantastic job. Jenny had commented that this was the first conference she had lasted the entire duration, so that says a lot about the pacing and content of PuppetCamp. I felt the same way, every session was incredibly engaging, and how Andrew had setup the democratic and chaotic Open Sessions was very impressive. Lets put it this way, I even got up there and pitched a topic, which is something I would have never done. Hurray for me stepping outside of my comfort zone!

Warning: side topic!

Now that I've had the weekend to google all the cool technologies I was exposed too, I'm also reminded why I really like having a FreeBSD server at my disposal. They had talked about CouchDB, so on a whim I did a ~> cd /usr/ports /usr/ports> make search name=couchdb Port: couchdb-0.9.0_1,1 Path: /usr/ports/databases/couchdb Info: A document database server, accessible via a RESTful JSON API Maint: till@php.net B-deps: ca_root_nss-3.11.9_2 curl-7.19.6_1 erlang-lite-r13b01_6,1 gettext-0.17_1 gmake-3.81_3 icu-3.8.1_2 libiconv-1.13.1 libtool-2.2.6a nspr-4.8 perl-5.8.9_3 spidermonkey-1.7.0 R-deps: ca_root_nss-3.11.9_2 curl-7.19.6_1 erlang-lite-r13b01_6,1 gettext-0.17_1 gmake-3.81_3 icu-3.8.1_2 libiconv-1.13.1 libtool-2.2.6a nspr-4.8 perl-5.8.9_3 spidermonkey-1.7.0 WWW: http://couchdb.org/ Port: py26-simplecouchdb-0.9.26 Path: /usr/ports/databases/py-simplecouchdb Info: Simple Librairy to Allow Python Applicationto Use CouchDB Maint: wenheping@gmail.com B-deps: py26-httplib2-0.5.0 py26-py-restclient-1.3.2 py26-setuptools-0.6c9 python26-2.6.2_3 R-deps: py26-httplib2-0.5.0 py26-py-restclient-1.3.2 py26-setuptools-0.6c9 python26-2.6.2_3 WWW: http://code.google.com/p/py-simplecouchdb/ I did a 'make install', and I had a cool little couchdb up and running. What is also cool is FreeBSD likes to give you very helpful information when you install something. For example, this is what is printed out when you install the CouchDB port: ===> COMPATIBILITY NOTE: CouchDB is still pre-stable; between 0.8 and 0.9 the database format changed which breaks BC. In current trunk, the format changed again, so please double-check in case you are updating an existing installation. More info: * http://wiki.apache.org/couchdb/Breaking_changes?action=show&redirect=BreakingChanges * http://wiki.apache.org/couchdb/BreakingChangesUpdateTrunkTo0Dot9 See, isn't that helpful? Best of all, I didn't have to enable additional repositories, or fetch the src manually, and its dependencies and then figure out how to run the right configure script flags... FreeBSD makes it easy, and since it automatically uses what you already have with what is required, its an incredibly stable build. Removing it is pretty simple as well, just: > pkg_deinstall -R couchdb ---> Deinstalling 'couchdb-0.9.0_1,1' ---> Deinstalling 'erlang-lite-r13b02,1' [Updating the pkgdb in /var/db/pkg ... - 118 packages found (-1 +0) (...) done] ---> Deinstalling 'curl-7.19.6_1' [Updating the pkgdb in /var/db/pkg ... - 117 packages found (-1 +0) (...) done] ---> Deinstalling 'ca_root_nss-3.11.9_2' ---> Deinstalling 'spidermonkey-1.7.0' ---> Deinstalling 'nspr-4.8' [Updating the pkgdb in /var/db/pkg ... - 116 packages found (-1 +0) (...) done] ---> Deinstalling 'gmake-3.81_3' [Updating the pkgdb in /var/db/pkg ... - 115 packages found (-1 +0) (...) done] ---> Deinstalling 'perl-threaded-5.8.9_3' [Updating the pkgdb in /var/db/pkg ... - 114 packages found (-1 +0) (...) done] ---> Deinstalling 'gettext-0.17_1' ---> Deinstalling 'libiconv-1.13.1' ---> Deinstalling 'icu-3.8.1_2' ---> Deinstalling 'libtool-2.2.6a' ** Listing the failed packages (-:ignored / *:skipped / !:failed) ! curl-7.19.6_1 (pkg_delete failed) ! ca_root_nss-3.11.9_2 (pkg_delete failed) ! perl-threaded-5.8.9_3 (pkg_delete failed) ! gettext-0.17_1 (pkg_delete failed) ! libiconv-1.13.1 (pkg_delete failed) This does a upwards recursive dependency removal. Also, if one dependency is relied on by another, it wont get removed. Like, if Perl58 was a dependency of a package, it wouldn't be removed if perl58 is used by many other packages. This is smart. So, above, the packages that failed to deinstall where ones that are required dependencies of other installed packages. Speaking of package management; have you ever installed something that ended up having a few dozen dependencies, then you want to uninstall that package with a "rpm -e cba8", or something equivalent, but what about all the other cruft that came along with it? You would have to keep track of each dependency, and specify all of them and hope you don't break another program. FreeBSD has a few tools to do this, one in particular, portmaster can remove all ports that were once a dependency but no longer used: > portmaster -s Information for neon28-0.28.4: Comment: An HTTP and WebDAV client library for Unix systems ===>>> neon28-0.28.4 is no longer depended on, delete? [n] y ===>>> Delete old and new distfiles for www/neon28 without prompting? [n] y ===>>> Running pkg_delete -f neon28-0.28.4 Information for rubygem-actionwebservice-1.2.6: ... I ended up removing 4 packages that were no longer used. CentOS and RHEL are the larger Puppet consumers, I'm still a big proponent for FreeBSD, and at work, it has allowed me to quickly build an Apache + Puppet + RubyPassenger/mod_rack stack with the minimal dependencies installed. So, the puppet server is still pretty lean, which means updates are smaller and faster. It still surprises me that its relatively unknown, even though Netcraft always has it listed in the top domains with the best uptime and consistently growing over the years. Why do I feel like an AmigaOS fan sometimes? Hmm, it is sort of weird that this turned into a FreeBSD ports management entry Okay, final word: PuppetCamp09 was Freaking awesome. There were a lot of smart developers and sysadmins there. We even got a very cool git howto, which I found useful. It was very diverse, which is strange for a conference based on one project in particular.

mike Geekyness FreeBSD, Jenny, kickassery, port management, Puppet, Ruby

PC-BSD is a nice mesh between FreeBSD and a ready to use Desktop (which uses about 6GB of disk space). It is based on FreeBSD 7.2, so it has all the cool features of the latest release. Best of all, without ANY additional configuration, I was able to:

• Use the official FreeBSD nVidia driver for hardware acceleration
• Watch clips on YouTube (with flashplayer)
• Play back all sorts of media types like mp3's, divx, mpegs,wmv,qt...
• use ZFS
• Create and edit documents with the latest OpenOffice 3.1
• Browse the web with Firefox 3.5
• Create VM's with VirtualBox

Plus, if there wasn't a PBI package for what I wanted, I could still use FreeBSD's pkg_add or, cd to /usr/ports and make one. I would say, that's pretty impressive for a commercially supported Unix platform. Otherwise, it is very much like Fedora or Ubuntu, where it has an update manager (updates PBI's and the system), network manager, helpful tutorials, and for once (for FreeBSD at least) a full blown X11/QT graphical installer. FreeBSD has always had a simple ncurses installer, which I like, but it tends to frighten a lot of people who are used to GUI installers. Once strange thing it does is place all PC-BSD binaries in /usr/PCBSD. I guess this is to remain independent and out of the way of the base FreeBSD binaries, as well as /usr/local, which is the normal prefix for all Ports. To wrap it up; my initial impression of PC-BSD is a positive one. I like how I could use the FreeBSD ports and package system and it did not conflict with the PC-BSD packages that were installed. I like the installer, and the storage options at install time (UFS2+SU, or UFS2+Journal, Encrypted swap...). With all OS's, it normally takes a few weeks of using it to see its weaknesses, so I'm sure PC-BSD has some issues waiting to pop up. The only one I see right now is that KDE4 is the default GUI, and I prefer Gnome. I could install it, but it wasn't an up front, as all of PC-BSD's install tools are written in QT. The initial X setup tool was pretty slick, and it worked with my picky laptop.

mike Geekyness Flash, FreeBSD, kickassery, OS Review, PC-BSD

FreeBSD 7.x has been using version 6 of ZFS, and originally only 8.0 was going to have the newly updated ZFS version: 13. Last week the core team MFC'd (Merge From Current) the ZFS updates to 7.2, so I cvsup'd and re-build my server's kernel and world ( with a simple "make buildworld && make buildkernel && make installworld && make installkernel ), rebooted, and now I have the latest ZFS version running:

[root@server ~]> zpool upgrade -v
This system is currently running ZFS pool version 13.

The following versions are supported:

VER  DESCRIPTION
---  --------------------------------------------------------
 1   Initial ZFS version
 2   Ditto blocks (replicated metadata)
 3   Hot spares and double parity RAID-Z
 4   zpool history
 5   Compression using the gzip algorithm
 6   bootfs pool property
 7   Separate intent log devices
 8   Delegated administration
 9   refquota and refreservation properties
 10  Cache devices
 11  Improved scrub performance
 12  Snapshot properties
 13  snapused property
For more information on a particular version, including supported releases, see:

http://www.opensolaris.org/os/community/zfs/version/N

Where 'N' is the version number.

mike Geekyness FreeBSD, ZFS

Over the past year I've had a slight resurgence in playing the guitar. Owen likes music, especially the guitar, and I've threatened a few times to dust off the electric guitar and really make some noise. After listening to a substantial amount of Pink Floyd, I got the urge (after 10 years of not playing) to pull everything down and get it all set up. I then hit a very unfortunate snag with my old Jackson JS20, the nut had chipped. This prevented the low E string from staying in place. The output jack was also broken, and as I recall, it was one of the reasons I stopped playing that guitar (and all together). I mean, there were other factors, but having your guitar ground out randomly was a great deterrent from playing. So I had to do two things: get a new nut, and a new 1/8th" jack; replace both respective pieces. The 1/8" mono jack was easy, I stopped by Radio Shack (besides being a cell phone store, they still amazingly stock electrical components) and picked up a few 1/8" STEREO jacks. Why Stereo and not mono? Well, I read that it helped provide a tighter fit for the plug, and it certainly does. It was a killer idea, glad I stumbled across that. Getting a nut proved to be a little more difficult. I drove to Guitar Center in Concord, talked to a sales associate there, and they gave me a new graphite nut. I came back home, and to my dismay, it was for a Les Paul, and even though it fit in place of the old, the channels for the strings were off, and the strings kept popping out. Disappointed, had to wait until I got back from Alaska before I would be able to make it back out to Guitar Center. When I went back to return the nut, another sales associate said that they didn't have a nut for my guitar, and it would have to be special ordered from Jackson. He also said he'd call them Monday, and call me back... he never did, even after I called and left a message twice. What a slacker. So, I started hitting up the local music shops here in Antioch where I finally ended up at Music and More. Bill, the spacey but very helpful technician offered to make one for me out of bone, and at this point I didn't care if it was carved out of my own hip bone, I was worn down from not getting anywhere. Well, after a few days of missing each other and him leaving a few messages, we finally met back up and he gave me a new white nut that he had lying around his house. Sweet! the color was different, my stock nut was black, but it doesn't really matter to me.

The repair of both parts was a lot of fun, and after all is said and done I'm happy with the results. I don't have any issues with the guitar cutting out, it plays well, and the nut is doing a great job of holding the strings in place. The guitar itself is decent, though I'd still really like to get a new one. After switching back and forth between the Jackson and the Yamaha SE250 (not shown), there are aspects of both of them that I like, and dislike. The Jackson has an extra pickup, so there is more tonal variation, the body is a little nicer, and I've spent a few years wearing down the frets with daily use (until I stopped that is). It sounds nice. However, I don't like the placement of the jack, its more prone to breaking (this happened to me once with my old bass guitar, pretty much ruined it). The Yamaha sounds like crap, and its harder to get it tuned (and stay in tune), but the neck is nice and I prefer the placement of the jack. The neck has more of a "C" shape, while the Jackson has a "[" shape, and I do find it a little more difficult to play on. I'll soon have to move all this out to the garage, I'm testing Michele's patience with all paraphernalia lying around. The guitar, the petal, the cables, the sheet music, its all cluttering up the front room.

mike Geekyness guitar repair, Jackson JS20