Hooked on Git (Hooks)

A few years ago, I made the switch from older source code control systems, to git. Yes, yes, I know. I used CVS for far too long. In truth, I'd been using git for quite a while on a variety of projects, but my own code was all still buried in a CVS archive. I found a tool to convert the repositories, while maintaining the history, and made the switch.

I still miss inline $Log$ and $Id$ but I'm over that. Since switching to git, I've had a few occasions to use githooks to capture a git push to the server. Mostly this has been about doing simple updates or sending emails. However, I now maintain all of my DNS configuration as a git repo. I also have a collection of Ruby scripts which process the data (either a zone file or a JSON file with basic zone data) and produce the master and slave Bind configurations.

Initially, these steps were decoupled, but eventually I decided to join the two so that a push to the master branch would also force the DNS infrastructure to be rebuilt and automatically deployed to the various name servers.

This has worked quite well, but it's not without it's difficulties. For some of the git-hooks, I have dumped a file in a certain directory and allowed for a cron task to notice the new file and do the appropriate legwork. This avoids trying to run complex processes as the git user and also avoids delaying the git push for too long. However, it's not particularly synchronous. I run the cron tasks about every fifteen minutes, which means there is a lag of fifteen minutes (worst case) from the git push until the data is deployed, and also it means I'm running the check script 96 times a day whereas changes are quite infrequent.

I've been thinking of either creating a Resque system and Redis interface, so that the git-hook just submits something to the Resque job queue via a Redis submission, or else I will code up some sort of job submission mechanism, to do something similar.

What I'd like to be able to do in the git-hook would be to run a command such as:

$ job-submit update-dns

This would either add an "update-dns" job to a generic queue or else it would add some sort of timestamp or equivalent to the "update-dns" queue. The idea would be to have something which consumed from that queue and ran the appropriate script. Perhaps looking for update-dns.rb or update-dns.sh in a specific directory, and executing it. I'm not sure I need something as advanced and complex as Resque but it seems silly to reinvent the wheel.

I'd also like to be able to do something which was multi-host as there are times when I'd like to do more advanced jobs other than just things called from a git hook. I think for now, however, I'll code something up to use Redis and limit the scope to local jobs only.

How to Ensure Your Heating System is Linked to an Atomic Clock

Quite some time ago, probably like a lot of other people, I converted my home heating system controller to an Atmel AVR circuit. For a whole lot of legacy reasons, I haven't tended to use Arduino boards for this. I have a selection of Atmel ATmega8 designs for previous projects, and I have a collection of etched PC boards which I reuse for projects such as this. Some day, I'll run out of etched boards and I'll switch to Arduino boards. If for no other reason than they are surface mount unlike my own designs which are through-hole. Also, assembly is not required! However that, like everything else, is a project for another day.

First off, it's worth mentioning how the system actually works, out of the box. My heating system is the of the OFCH persuasion, or "oil-fired central heating." For those of you not in a cold climate, this involves a boiler and burner unit, which heats a recirculating closed water system, connected to a collection of radiators around the house. The burner has a blower and oil pump (as well as an electronic ignition). This is fed via a thermostat on the boiler to make sure the recirculating water is at a certain temperature. Above that temperature, the burner shuts off. There is also a pump which moves the hot water around the system and through the radiators. Along with this are two electrically-operated valves. One opens the system to the radiators downstairs and one for the upstairs radiators. Generally you never run the burner without the pump running, as this would just heat the stagnant water inside the boiler, to no effect. The thermostat makes sure that the water inside the boiler never gets too hot, cutting off the burner if and when that happens. Generally the return feed from the house will be below the cutoff so the burner operates. However, eventually the return feed will exceed the thermostat and the burner will switch off, leaving the pump to circulate the water until it drops below the low temperature threshold (hysteresis and all that). With both electric valves off, the boiler will heat the hot water system in the house by way of a heat exchanger in the hot water cylinder. Generally though, I run the burner, pump and downstairs valve together.

I tend to phase in my projects rather than try to swallow the elephant in one bite. The first phase was to remove the existing (dumb) unit and replace it with a bank of four solid-state relays. These Omron units are quite good. They operate off a 5 volt input so they can be connected directly to the Atmel chip (I use 5 volt Atmels - your mileage may vary). In order to isolate the AC power from the low-voltage systems, I mounted the four relays in a wooden box with a plexiglass front panel. As this was enough of a project to get me started, I then took a five volt power supply and connected it to another dumb timer. I fed the output to three of the four relays, ignoring the upstairs system to start with, as it isn't used very often. At the appointed time, the dumb timer would energize, power up the 5 volt PSU which provided an input signal to the three relays, thus turning on the central heating system. The phase one version of things operated for quite some time!

The second phase introduced the Atmel circuit. I decided to simplify the programming and settled on a sixteen entry program table. Each entry specifies a start time (HH:MM), a duration in minutes (up to 255 minutes), a day-of-week mask, and a operation byte. The operation was one of {H,H+D,H+U,H+D+U}{I} where H was for hot water (just the burner and pump), D was for downstairs (burner, pump and downstairs electric valve), U was for upstairs and I was for an electric immersion heater, filed under TBD. I used a serial port to talk to the Atmel board, and wired that to the house server, temporarily. I built a sealed lead-acid (SLA) charging circuit, and mounted the Atmel board with the 12 volt SLA in another wooden box with a plexiglass cover. I put a 220 volt to ~15 volt AC transformer into its own sealed box and wall-mounted that as well. In addition, I put a separate unit with two circuit breakers on the wall. One breaker feeds power to the Atmel transformer and one to the bank of relays. The code keeps track of the exact time of day, and day of week.

This worked extremely well, but changes to the program schedule, or requests for a quick "burn" of heat, were tricky. There are times when you just want to turn on the heating system for an hour, without updating the program. I supported this option in the firmware, but it involved having to use /cu/ or /tip/ to connect to the serial port and send the command.

Some time after this, I came across a Grandstream Serial Concentrator from Perle. It has 24 serial ports available at the front via RJ45 sockets. You can configure the device in a number of ways, but for this I configured it as an outgoing raw port. It is set up to listen on TCP port 5024 and relay data to the serial port, and vice versa. If you happen upon one of these devices in your travels, they are well worth adding to your equipment rack. This allowed me to interface the Atmel board with the house network, and from there, I developed a small web server which allows me to control the heating and adjust the program. More on that, in a later posting.

As my main server (an HP Proliant Microserver naturally) runs FreeBSD and also is an NTP client, so it has a very accurate view of the date and time. I have a cron task which is responsible for updating the day-of-week and time on the Atmel board. This takes care of two things: Firstly, clock drift is eliminated as the time is updated every hour. Secondly, I don't need to code for the abomination known as "daylight savings time" as the cron task will reset the hour shortly after the clocks change.

There is a smug self-satisfaction from hearing the heating system fire to life at exactly 0730 in the morning, knowing that the clock in the heating system had been corrected using an Atomic Clock reference, a mere 30 minutes earlier.

The system isn't without it's challenges, and I'll cover those in a later posting.

Apple and the Fields of Athenry

I hate to rain on the Apple parade, and in a lot of ways it's good news for Athenry, but some of the hyperbole, particularly the political kind, is irritating. First off, this is a data centre. We can assume land costs are on the order of $5 million. Construction costs for shelled out DCs are pretty low, in comparison to office buildings. The vast majority of the $850 million budget, will be spent on servers. Rough estimates say they might deploy up to one million of them. List prices for DC servers run from $1,000 on up. Apple will end up paying a lot less than that. Either way, unless Athenry makes computers, most of that money is going elsewhere.

Intel in Leixlip could do well. Server manufacturers around the world are no doubt already working on quotes. In terms of staffing, the facility will need 24/7 care. If they're quoting staff numbers in and around 100, that means four shifts of 25 DCOps (or data centre operations) staff, whose job it is to install new equipment and replace faulty servers and hard drives. You can be trained in how to do this in a matter of weeks. You don't need motorways to Dublin or Shannon, because your job doesn't involve travel. These types of facilities are called "lights out" facilities because they run in total darkness (unless DC Ops staff are on the floor, obviously). Why waste electricity providing lighting for banks of computers? Some "artists impressions" of the data centres show them with large windows. Obviously the artists in question have never looked at a photograph of a real data centre, which can often look like a high security prison. Windows don't help with heat efficiency or security.

The real reason Athenry is ideal for Apple is due to the cold weather, the ESB fibre to Dublin, along with the motorway fibre, the railway fibre, and the imminent arrival of a new transatlantic cable, via Belmullet. To understand the energy economics a bit better, the quoted 360MW of power used by a DC such as this is mostly transformed to heat, by thousands upon thousands of servers. For every kilowatt of power that a server consumes, another 200 watts is needed to remove the heat produced. That figure can go as high as 600 or even 800 watts. Power is one of the biggest costs in your average data centre, and cooling is a big headache. The ratio of total power used in comparison with power used by servers, is called Power Usage Effectiveness or PUE, and Apply will be gunning for a PUE in the 1.2 range. This will mean 300MW of power for around a million servers, and 60MW of cooling. By choosing rural locations where land costs are low and cold winds are plentiful, you can reduce the cooling bill and bring that all-important PUE down. It also helps if there are large amounts of renewable energy sources around, but wind energy while great, is unreliable. We do get calm days in the west of Ireland, and if it's hot and calm, the PUE can start to creep up. Luckily for Apple, those days are rare.

Wind energy is on the order of the cube of the wind speed. Simply put, 10km/h of wind might produce 1000 units of energy, but 50km/h will produce 125,000 units. Or 125 times the energy, assuming that 10km/h is strong enough to rotate the blades of a wind turbine, and 50km/h isn't too strong. While that's great, and we do get our fair share of strong winds, looking at it from the DC perspective, if the wind drops from 50km/h to 40km/h and those of us in exposed areas breathe a sigh of relief, the power produced will halve. That means having to get 50% of your power from traditional sources just due to a drop of wind speed from 50 to 40. By the time it drops back to 10km/h, almost all of your power needs are coming off the national grid. But let's not quibble. Any opportunity to reduce our dependency on coal, to reduce the amount of CO2 we produce, and to reduce the impact and need for the unsightly Moneypoint Power Station, is a good thing, even if a lot of the green energy we produce is being used for iTunes downloads.

In reality, Apple is known as a good employer. They've always treated their staff well, and the one hundred DC Ops staff can look forward to a range of benefits and perks which may make their neighbours quite jealous. But unfortunately this isn't a windfall for Athenry in and of itself. The people of Athenry could use it to their benefit though, by trying to attract other players into the market. If Apple is there, then there is a guarantee of high-bandwidth IP connectivity to the East and to the West. In addition, wind farm developers may choose to locate in the area, and the national grid will have to make sure that the area is well-served by traditional power. All of this makes Athenry a good choice for other data centre operators who to date have chosen the "T-50" (a highly-interconnected area in West Dublin, and home to both Google and Microsoft data centres). Good for connectivity, but not as good for land prices and natural cooling.

All in all, it's great news for Apple, Athenry and the west in general. But be careful when people bandy about FDI figures of $850 million. Oh, and if you are reading commentary about Apple in Athenry, and the article mentions the Google and Microsoft DCs in Dublin but doesn't mention Amazon, then they are uninformed and should not be trusted.. Amazon are the 800lb gorilla of cloud computing, and the best-kept secret in cloud. They operate six enormous facilities in south Dublin, which is nothing in comparison to their facilities in US-East.

Leaky Windows and Launch Dates

So, last September while on board Nikea in Greece, I decided it was high time the Achilles 24 saw the sea. She has been languishing beside the house for a staggering ten years (where does it go, etc). These days, I mostly seem to sail other peoples boats, which, by the way, is a great way to sail. But after a couple of days of pottering on board Nikea on my own, I found that I was very taken with single-handed sailing. No, I'm not planning on doing the Vendee Globe, but I do like the idea of spending a few nights on the hook in Greatmans Bay in Galway, or in Kilmurvy or even as far as Inisboffin or Westport.

So, a plan was formed. Officially the Achilles has been up for sale for most of those ten years, but boat buyers are a strange and rare breed. Particularly in a recession. They come along sporadically, and always seem to be looking for a perfect little boat, at a knock-down price. That's something we'd all love to find. Due to some leaky windows, hatches and other above-the-waterline entry points, rainwater collects in the boat. For this reason, everything which isn't bolted down (and even some things that are), were lifted out and stored in the garage and attic. This makes it harder to sell the boat because she looks less like a  boat and more like a hollowed-out fibreglass hull. So, slowly last year, and with more and more emphasis this year, the Achilles is being prepared for the water. The decks were power-hosed and theninterior was varnished. The engine is off to see the Engine Guy, and the shopping list of new parts grows ever-longer. Who knows if I will get as much single-handed sailing as I intend, but the process has started well. If nothing else, she'll be in a better shape for sale by mid-season.

I'll be keeping this blog relatively up to date on my launch activities, and it will remain to be seen if my optimistic launch date of April 26th actually happens.

Hopes for a true film collective?

Unfortunately, this will probably end up as a very long post. It will also probably end up as being quite polemic, and may upset some people, but there you go.

Recently, the Irish Independent talked about the lack of funding for Irish film. While that is a laudable debate, unfortunately the article echoed recent comments by Siun Ni Rannallagh regarding additional funding for Ardmore Studios. This, however, is not a reasonable request. First off, the studio has been well-funded by the Irish taxpayer throughout it's less-than-illustrious history, and recent developments as documented by Tom Dowling, show that little has changed. It is no secret that the Vikings TV series went to Ashford Studios. Up until that move, Morgan O'Sullivan was a strong proponent of the studio. It is anyone's guess why this happened. Anyway, I'm not here to bitch and moan about Ardmore. I wish them every success, but I would not like to see taxpayer money poured on top of existing investments, when hospital waiting lists are increasing, and the rest of the economy is in the toilet. It's not like the owners of the studio are short a few bob. If they have grand plans for the studio, let them fund those plans through traditional means.

I guess the bigger (and more reactionary) concern of mine, is the overall state of the industry. We are led to believe that the Irish film industry is worth tens or even hundreds of millions a year in direct and indirect income. However, our catalogue of releases shows that the vast majority of Irish films don't recoup their budgets, much less make a profit. A lot of films don't even see a theatrical release, beyond a token gesture in a handful of screens. Let me be clear, though: Irish film is as important as it ever was. We need to see these films. We need these films to be made. I don't dispute that for a second. What I dispute is the notion that we have an industry. By any benchmark, it is not an industry. It cannot sustain itself without FDI and state funding.

If we look objectively and impartially, we could see an industry which could support perhaps ten or twenty heads of department (you may dispute those numbers, but I doubt very much if you'd claim they were off by an order of magnitude). In this context, I mean directors, cinematographers, sound mixers, editors, and so on. While each of these may also have five or six people who work for or with them, the numbers are underwhelming. The situation is slightly better for actors and writers, but most actors barely subsist. They do one or two TV commercials over their careers, they do a lot of profit-share theatre (which also has its issues), and a large number of free, student films. Contrast this with the number of film schools in Ireland, and the sheer volume of film students who graduate every year, fully expecting a career in film. Like many people who run a small production company, I am inundated with emails from students seeking unpaid intern jobs. This is just unsustainable.

As someone who has directed a number of short films, and someone who threatens to shoot something long-form in the near future, I am conflicted by the need to support people in their chosen profession, and the absolute dearth of funding for film. Yes, the Irish Film Board does a good job, but rather than cultivate independent cinema, they seem to act as a gateway to it. A sort of self-censor for Irish film. Their short film budgets are excellent, but they only fund a handful of films. Likewise, their feature film supports are superb, but it has reached the stage where an Irish film can't get made without the implicit approval of the Board. This is not good. Yes, there are films like Charlie Casanova, but Terry's film is the exception. Terry himself got sick of waiting for Film Board money and decided to go it alone. It wasn't easy, and his tireless efforts on behalf of that film, did a lot for it. It also helped that it is a good but controversial film. I know of other film-makers who have also tired of shaking the IFB money tree, and have found themselves between the proverbial rock, and hard place.

Charlie Casanova could not have been made without the support of a large number of people who gave of their spare time, freely and willingly. I know Terry would have liked to have paid them, but given the choice between not paying them and making the film, or not making the film, he rightly chose the former. As a film-maker, I have also relied on the "kindness of strangers" to help me make my films. But therein lies the rub. They're not my films. Everyone who worked on the film could claim ownership and credit for it. I was the one who paid for the lunches, the accommodation, the equipment hire, transport, and all of the unavoidable costs. But it is a collaborative effort. The film doesn't stand or fall purely on the efforts of the director or the producer. It requires everyone to bring their best work to the endeavour. Many times, while making zero-budget films, I have been asked (sometimes quite strongly) to make a payment to crew members. If I could afford it, I would have no compunction about agreeing. But there's a down side to that, too. If I had the funding to pay for a cinematographer or an actor, I would choose the best available for the money, seeing as money was no object. If someone wants to start out in this insane business, and works their way into a second-assistant camera position, they will spend a long time looking for work as a cinematographer, while their mentors are also looking for work. Their only hope is to work on an independent, no-budget short or feature.

There, I've said it. I've put paid to the notion that people just starting out in film, can earn the same as people who have been doing it for years. They have to prove their worth, and in film there aren't many opportunities to prove yourself. But there's another side to this. A more important facet of the debate. It is this cock-eyed notion that you became a film-maker to make a lot of money (if you did, I would suggest suing your career guidance teacher). Even at the top of your game, you can expect a life of boom and bust. Times when the film-gods smile upon you, and you are busy. Interspersed with times where, as one accomplished film-maker said, "I have too much life in my work-life balance." The truth is, you decided on a career in film, knowing only too well that your chances of making a living exclusively from film, are quite slim. Particularly at the upper end of the scale. As a second-second assistant director, you may be able to keep your calendar full, between Morgan O'Sullivan and RTE, with TV3 also starting to fund production. But take, for example, RTE's most successful independent production in many years; "Love/Hate." One writer, and one director. If you were hoping for a gig like that to pay the mortgage, you are in trouble.

So, if you acknowledge that unless you're one of a very small cadre of full-time workers in the Irish film industry, you also acknowledge that you will need to supplement your income, elsewhere. If you don't believe me, research the salary figures for members of the Directors Guild of America, bearing in mind that to become a DGA member is quite a difficult process. Likewise, look at the average annual income of SAG actors, again bearing in mind how hard it is to join that particular union. The situation in this country is much worse.

Where does this leave film? Having lived and worked in Northern California, and also in London, I was pleased to be part of a film community in both locations. A community that worked together to make films. In some instances, such as the London Film-Makers Co-Op, also pooling resources such as cameras and edit suites. I know that the Galway Film Centre and Filmbase both attempt to provide this kind of "collective" experience, but I wonder how successful they really are at encouraging film-makers to just get out there and make films?

What I'm proposing, is to try and set up a similar collective, but without the offices, buildings, membership fee or overheads. Perhaps people help you direct your first feature? But bear in mind that you will have to drop everything and "swing boom" for their projects, when the time comes. A collective which is open to directors, cinematographers, actors, writers, producers, editors, animators, and whoever else wants to make more films. If you've had enough of rejection letters and failed funding requests, maybe this is for you? But don't expect to be paid.

I know this will raise the ire of many, particularly the unions, who see an endeavour like this as an opportunity to circumvent their daily minimums. However, if even the well-funded films aren't making a profit, where is the exploitation in decided to make films for the sake of making films? It is quite easy to write a manifesto and produce a contract which stipulates that any profits for the film will be divided amongst the people involved, but that gives weight to the lie that the film will turn a profit. If it makes people happy, then so be it, we can let the lawyers define the terms.

The other side to this, which is rarely acknowledged, is the fact that the only way to get better at making films, is to make films. Odd, that. If we all sit around waiting to win the IFB lottery, if the day ever comes, we'll find ourselves unprepared for the experience. By filling our idle moments with the practice of making films, we become better film-makers as a result.

My intent here isn't to extend some sort of job-bridge scheme, or exploit students. Far from it. The intent is to enable people to make their own films, by helping others with theirs. The intent is to get back to the real reason why we make films. The intent, really, is to find out if I am alone in this, or if there is a constituency of disenfranchised film-makers, who are itching to get out there, and make films. Is that you? Or maybe you're repulsed by the idea of free labour? Either way, I would love to hear your thoughts and comments. Either this is an idea which should die on the vine, or it's the start of something.