Have I finally fooled Google?

I went to search for Douglas Kennedy, and was directed to his page in French. Since Cinq Jours is a translation "de l'American," there are one of two possibilities. Either the official page of Mr Kennedy is in French, or I have successfully convinced Google I'm not interested in the book "Five Days" from which this is translated. I suspect Mr Kennedy has a good reputation in Europe and puts his french face forward, but I may have won, quoi sais-je, moi?

Defining Pi

Sometimes, when working in Scheme, I'm surprised that pi is not defined. I'd define it myself, but to what precision? My current method is to exploit Euler's identity to get the basic precision supported by log. Is there a cleaner way to initialise this?

 (define pi (imag-part (log -1)))

Information Hiding

(I'm not going to talk about steganography.) There's an idea in object oriented design that suggests that information hiding is part of the success of encapsulation. Having worked in OO languages that do not have the notion of private/public interfaces (Smalltalk is a great example) the way Java or C++ do, I am sometimes skeptical of the benefits of this. I believe I see a point in programmatically specifying your level of abstraction and sticking to it. You worked hard to make the abstraction layer, and it would be useful to be able to see it used, and perhaps know where it was misused or bypassed. On the other hand, I think enforcement of this leads to a huge level of headaches, and changing the interface after the design is daunting, since the compiler will likely throw a fit. In conclusion, since it interferes with redesign and rethinking, for me it's a misfeature.

I always thought it was silly if you have the source to 'hide' the implementation. The pointer to implementation pattern in C++ is the ultimate expression of this fetish, trying to move the guts out of the header (since there is a weird amount of required information in the header files) and into the source, where I guess it can be compiled and hidden in object code. Here I disagree with both the sentiment and the technique.

I finally get it

I have spent too long wondering why things were harder in apt than I wished they were. Mainly this involves my constant complaint that a package could somehow install without the documentation, and not knowing if it's possible to configure synaptic to honor 'suggests' hints, and automatically install -dev and -doc for any selection. I now finally understand apt-get source and apt-get build-dep, which go a long way toward allaying my problems.

In the past, what I had done was something similar to

apt-get install package

and if package was unfortunately lagging (sbcl is a prime example where the repositories kept an old version, but anything under active development is going to have this issue) I would clone a git repo and build myself. Now, the development tree assumes, you know, that you have current versions of all the development tools, and all the development headers for all the dependencies. Since I had run off the apt track, I regressed ten years back and started package hunting, configure, scratch head, read error message, find next dependency, configure, scratch head...

Remember, this is the problem these package managers were supposed to solve. So I now see how it is solved.

apt-get source package

(does not need privileges) downloads the patched sources to build the binary version in dpkg. This is great, you can modify the options a little, but still let dpkg know you have a version installed for this (to prevent a stupid binary being installed into /usr when the live version is in /usr/bin). However, it still gives me the stable/stale version. So this is where the next part comes in handy.

apt-get build-dep package

will make sure that you have a workable toolchain to satisfy configure's needs. Now I clone the latest and greatest from git or svn, and apt-get build-dep package, and then I'm having nearly an 100% success ratio in building. I always knew the debian tools were the best, I had only forgotten why.

Ventra, your wallet, and you.

The CTA recently rolled out it's new payment system, Ventra. This is an RF chip used in major credit cards that allows you to tap your card above the turnstile reader and go right in. This has the added benefit of flexibility, in that tourists or anyone need not buy a farecard, but can simple use their existing credit or debit card for fare payment.

This is great, except that you may three or four of these cards in your wallet. Looking over my recent activity, at least on train ride was paid by my bank atm card, and another was paid by my america express card. I guess simply holding my wallet up to the reader will not be sufficient, and instead I will need to carefully hold the ventra card alone up to the reader when entering a station. For the most part, it seems that this has been working fairly well, since I have less than a 20% error rate for cards essentially next to each other in my wallet.

This is only a quirk for me, since I don't have a monthly pass (which wouldn't have been used when I pay directly with a different card) and am not yet using the pretax fare payments through my employer. I can see this catching a number of people by surprise, and it's a departure from the previous chicago card chip, which would not accept outside payments, and would work right inside your wallet.

Nights

I now have on average 3.5 nights per week of free time, where I can't go out since the sun's down (years in San Diego have persuaded me that the sun is the critical deciding factor for outdoor activity, years in Chicago have terrified me of the night), and have time to spend doing something meaningful. By meaningful, I intend not involving alcohol or /b/.

So, what have I been up to? I installed antiweb at beta-reduction, my vps at work. This was a simple case of (1) finding a CL implementation which works on CentOS with a nazi kernel eager to kill off any memory hogs -- in this case Clozure works marvels, SBCL fails to build for me, and building even CLisp proved troublesome, segfaulting the moment quicklisp was loaded; it's possible ecl is viable here, (2) install anti-web, (3) create /var/aw, (4) add a few dummy workers, in this case beta-reduction.com and iota.beta-reduction.com. What I'd love to do is investigate dynamic page generation, particularly using UCW. How workers handle cgi marshalling is yet beyond me, my light goals may be to implement simple pages in guile or ccl, with antiweb acting as a dispatcher.  Everything I learned at work about what a LAMP site does is completely overthrown by this setup, since for one hub and two workers each at 512GB Virtual memory, thus I have over 1.5TB of VM allocations, which aren't triggering OOM conditions. In contast, it seems most programs assume they can and will use the memory they allocate, and controlling mysql and php allocations for customers is maybe 1/8 of my job.

 I've started working again on 99 lisp problems, which keeps me busy this week. I'm surprised to look at previous times I started this and where I stopped.  I must be getting more proficient, to have made it several problems further. The best part is solving an intermediate problem in one or two lines using the last result. I never understood mapcan until I needed it. I think there's an internal timer in me, if a problem takes more than one hour to solve, I tend to move on. This time I made it to problem twenty-seven : given a list of items acting as a set, generate all possible sets of disjoint subsets of a specified size, as

(defun group (set sizes)
 ; something interesting here)

Also, this is the first week of classes at Roosevelt, so I think I will not be there for this fall semester. I may contact Dr's Pivarski and Urbina to see if this Spring makes more sense. It hurts to be 3 courses from a MS, but I can't afford more time to a degree without an exit plan. I also can't spend 36-48 hours per week at work and then take 1-2 nights from my free time to go to school without making Beth sad panda. I guess college might not have been my thing. (It really was, but what can you do?)

Still plowing through French and English readings. Just finished 'The Spectator Bird' and 'Bug-Jargal', moving into 'Dernier jour d'un condamné'. Before that was 'La Philosphie dans le Budoir' and a collection of poetry from 'Le Chat Noir', which was interesting, though I wish it had had more Allais. I read Villette, which I think is perhaps at least as good as Jane Eyre.

My Next Week, and my summer so far

Summer is winding down. In May, I started at WiredTree, with the expectation that after a few months I would be moving to night work. I just had the meeting with my manager, and Monday the 12th of August will be my last Nine to Five day in the office. I will begin my late work at 8PM on Wednesday the 14th. So my schedule will look something like Wednesday night to Friday night every week with Saturday nights also every other week. If it works out, I may not need to move my ballet tickets too much. If not, there's some flexibility in time off that I can take advantage of.

Training has been an interesting experience. I'm using linux in ways I never had, and dealing with much different aspects of the system. I'm seeing PHP up close, a language I never needed. I'm dealing with ill defined plugins to solve strangely ambitious problems, (I just want the page to load fast). I see the slime that inhabit the internet, smut peddlers, spammers, scammers, and people hawking little plastic pieces up close. I'm beginning to sympathise with them and understand their motivations. I may, gasp, end up revisiting Perl. Since I had always previously treated computers as really fast adding machines, the whole internet thing seems to have past me by without changing my habits too much. That's why my twitter account is quiet, my Facebook was deleted years ago, this blog gets a post when I feel like it. But I seem to pull the current emacs and sbcl branches at least once a week.

Unloaded!

Say what you will, but I think ZaReason's choice to give customers the option of an unloaded device is quite laudable. Would more companies were so enlightened.

Money and Art

Nina Paley had an interesting series of thoughts about value and the art world here. The pearl I found most striking: "Now I realize that collectors are the prime – or only – originators of high-end art value. Anyone can paint, but only a tiny elite can buy a painting for a million dollars."

Ah, Seneca

itaque cum legere non possis quantum habueris, satis est habere quantum legas. Since you cannot read all that you might have, it is enough to have as much as you read.

Week Three

After noting down a few reactions to my first week back at work, maybe it's a good idea to continue in that direction for a while. I finished my third week at work this Friday, and am enjoying a federal holiday off today. I've moved from filling out worksheets to explore the systems they support to starting to support their systems.

This week began with abuse, copyright, phishing and spam incident handling, and progressed by the end of the week to active user requests (the bread and butter of what I will be doing for some time). It was funny in that I was able to shut down a site for phishing on Thursday afternoon, and assist the admin in restoring it on Friday.

The only really funny part, is having to adjust to all these servers sitting in a datacenter in downtown chicago, set to local time for the admins scattered around the globe. At 1PM on Friday in the central US, it's half past midnight in Mumbai the next morning. Staring at timestamps gets to be great fun when you always have to ask the system what time it is to make any inferences about the past.

I have started to notice the sound of phones ringing, for a while I was under the impression that it seldom happened, since it was deafeningly silent for the most part in the office. As I get more of a handle on what happens there, I think I start to notice it happening more as well.

The danger of simple surface syntax hiding a complicated model beneath

Getting back into the python pool, I am often bitten by nuisance features, most of them having to do with 'readable' syntax... or just syntax at all.
For example, writing a gcd method today, I wanted to say:

while b is not 0:

Which has the rather readable quality of saying exactly what I would say if I were to explain what I wanted in English, and the subtle disadvantage that it relies on object identity rather than numerical equivalence. Then I had the unfortunate wakeup call that fairly small numbers are long in python, and that the difference of two longs is long, and that these upcast variable types are not downcast to int when they come into range again. Coming from a lisp world where numbers don't play these games, and in fact

 (type-of (- 10 10))

answers bit, this was a bit of a shock. So

1L - 1L = 0L

and

 0L is not 0

is true, which is really not what I meant! I had to read the error, saying something about division by long zero, which didn't seem clear at all to me, and figure out where in the 'simple' number handling system this problem was popping up. Good times.

Week One

So week one was at the same time exhilarating and exhausting. It's been a long time since I was expected to be in one place for more then a few hours at a time. It's been years since I had my supervisors nearby, so learning to behave again is a great experience.

They started three new people on the same day, and set us all to the same tasks. I have this inkling that there's a bit of selection going on here, but I think I may be paranoid. We all sit at our own terminals in the middle of the room, and step through various configuration and repair options.

I registered a domain name (my old one, djuber.com, has now been purchased by some uber-dj to promote his gigs) beta-reduction.com. My webserver is completely hosed after all the changes I had to make to the server in the course of the administration worksheets. On the plus side, the email works, so sending to me at that domain will work. I'm not horribly sad about the misconfiguration, since at its pinnacle the page displayed a nearly empty wordpress blog. My only regret is looking incompetent because my server doesn't work and I haven't rescued it. At least no one around me seems to be doing much better. When they say they're done, and I ask how they did something, it looks like they just cut a few corners.

I have gotten a little better at using find (I never used flags with that before) and fgrep.

Next on my list of conquests will be chewing through the rather long gawk manual (my pdf version is just shy of 500 pages, and the chapter "Practical awk Programs" begins on page 245). Awk and Sed have been tools I've been putting off learning to use for too long. Additionally, I want to get through the Advanced Bash Scripting Guide. Then I will be an admin's admin.

Go2Shell

I can't believe it took me this long to learn to adjust this, but I'm glad that the Go2Shell Finder extension supports launching iTerm rather than Terminal.app. This is happiness for me, though I don't much use Finder, since I spend more time in emacs using dired to handle file navigation.

Life and Code

First off, I've taken a job, starting immediately after my finals, at the beginning of May. Although I am thankful for the education I've received at Roosevelt (and especially that at UIC), it's time to move on. Partly this is due to not having sufficient VA funding left for another full year, and largely it's due to needing an income. Student loans seem very manageable while they're deferred, but who can say what how fun they'll be when it comes time to start paying them down. I'm ambitious to get those eliminated in as short a time as possible. My hesitation fifteen years ago to incur debt (when university education was perhaps a third of its current price) is being rewarded today by a higher sticker price, and perhaps a less rosy outlook.

Secondly, since the job I've taken involves linux hosting, I have been trying to brush up on the gentoo universe. I've used gentoo in the past, it's a wonderful system, rather like FreeBSD's ports system, or the wonderful Homebrew suite for OS X. The installation is a little bit tedious, since you need to do many of the parts by hand, but less troublesome than say, Linux From Scratch, since the guide is directed to getting you finished quickly, rather than exploring the intricacies of building binutils and libc in several stages. Instead, Gentoo provides a base image (the stage3 tarball) and allows you to build up the minimum bootable system from that, without rebuilding the entire toolchain. It's certainly not a point and click install, and you need to know how grub and fdisk work, but the tradeoff is that nothing you don't need is installed, which is excellent for servers. If you know the ten packages you want, and which features you'll use, the small number of dependencies is automatically fetched and built to suit your needs. Very clean, but not if this is your first (or second) linux system.

The interview went very well, either I know what I'm talking about, or they aren't particularly selective in entry-level staffing. Optimistically, I'll assume some middle ground between the two, that I'm sufficiently qualified for what they need, and apparently educable enough to move forward.

Code-wise,  I've been returning to Python after a year or two. This is stimulated a little by the cool things people continue to do in python, partly by its ubiquity in the world of automated system administration. There are still so many things which bother me about it on a linguistic level, but I think it's a good place to play. I'm trying to reinforce some of the smalltalk experience I had, where objects and their classes are nice friends to have, and high level is an understatement. It's a very different world from the code I did this spring for my algorithms class, which was a SAT solver in C++, but I basically wrote a C program with iostreams for input/output. That was fun, I used a fairly clever shallow copy and copy-on-write idea to speed up the program about five times, and reduce memory use to a constant level, depending on size. My initial, naive approach crashed on large problems after allocating blocks like a fool.

I checked out Code-Dojo, which allows for rapid test iteration. After last summers experience with Pharo Smalltalk, I have to say I am a confirmed unit test enthusiast. For Common Lisp I prefer fiveam, which has a pretty gentle learning curve. For Smalltalk, the support for SUnit is first class, and really part of any package: the packaging tools in Pharo automatically bundle tests with the code, and the Nautilus Browser has a handy jump to test feature that will, if needed, create a test case class, and populate it with a testTheMethodIWasJustLookingAt method, and change focus to the new test. Tests can be run from the browser or the TestRunner tool, and it's a great experience. In python, I've been leaning toward unittest, after a brief run in with py.test. Py.test was easier to get started with (just write a function beginning with the word test, and it's automatically discovered), but unittest seems more likely to be separable from the source code. My current setup is a vertically split pane in Emacs, with code on the left and tests on the right in separate files, and sometimes I'll leave the python process in a horizontal pane across the bottom (though this often annoys me too much when toggling buffers, so usually it's buried, and I just bring it into focus when running tests.)

So, in the last day, I rigged up a (probably inefficient) Sudoku problem solver, which is available on github, and build it bottom up either test first, or test immediately. I probably could have done a more efficient approach (I can't help thinking I check the whole puzzle way too many times), but it solves an empty puzzle in around 10s, and a puzzle from the paper rather quickly. Plus, it was challenging enough to force me to rediscover some of the warts in python, i.e. [0]*10 makes a list with 10 copies of 0 in it, but [[0]*10]*10 makes a list with ten references to the same list of zeros. Had a fun time failing tests with that bit of unexpected behavior. Also, I worked through some of the simple projects on code-dojo, the one hundred doors and anagram exercises. That was useful, and I hope to repeat it again. I once saw two people from Eighth Light doing live coding for the change counter exercise, one in Ruby, the other in Python, side by side, just making failing tests, and extending the code. Having first seen this in action, I came away from that experience wondering how I could go out and use this method in my programs, and have tried to approach projects with TDD in mind ever since.

Besides catching mistakes early, I enjoy the freedom it gives you to feel comfortable making substantial changes in the implementation while ensuring that the exposed interface is maintained. Verbose tests are much more friendly to me than a debugger, especially when the runtime and compiler aren't present in the debugger. GDB is a powerful tool, but looking at source, and touching it are separate affairs. In C, I find myself writing functions which are only useful inside the debugger, say, for formatting object inspect from a pointer (maybe there's a great way to automate this in gdb, but I haven't discovered it, yet).

I've looked at trying this in C++, but the Boost test system was overwhelming, and not exactly trivial to implement. In fact, until some well supported community best practice emerges in C++, I don't know that I'll use test suites there. The code overhead of implementing another class, populating it with tests, directing these to run in some manner, and instantiating it (does this go in a separate debug executable?) seems a little heavy, in a language environment where you are already drowning in details of the system, and the compiler backtraces are sometimes helpful, sometimes downright silly. Coming from Smalltalk and Lisp, the experience of highly syntaxy languages is a little jarring. I'm sure It's handy to be able to instantiate a dictionary literal, or access a slice of an array or list, but all these shortcuts glue together a little unpredictably (like the above list multiplication operator using shallow copies, rather than deep ones). Also, the major gain of lisp and smalltalk for me has been the environments. I can ask Pharo or Squeak where some bit of code is, and it will bring it right up for me. In emacs, with slime and sbcl (with source locations turned on), I can find every definition of any symbol with M-., and debug macroexpansions with C-c RET, and inspect and change objects in the debugger. It's not as pretty to look at as smalltalk, but the idea that the objects are alive, responsive, and ready to build your program is definitely invigorating. With Python, I think I'll need to see what the pdb or some other debugger offer me. It's definitely not the case that I can ask for the implementation details of list or dictionary from the system, and the python.org docs seem a little cursory when discussing the implementation details (linking module docs to source trees in git would be  wonderful, and I'm not the first to ask for that). Although it's open source, and mainly written in itself, finding the source of some method in python is not automatic.

One last gripe, and it's a small glitch for me, is when I try to execute a python script from a buffer in emacs, it defaults to python -i. It would be wonderful if I learned how to override this to read the shebang line and act appropriately (i.e. call python or python3 as specified at the top). That probably just involves a little digging in the configuration (is that rope or pymacs handling this behavior?). The emacs integration is a little less tight than slime, which really tended to make emacs a superpowered ide for lisp, rather than an editor with some shell out capacities.