20 Jan 2016
A few years ago, I made the switch from older source code control systems, to git.
Yes, yes, I know.
I used CVS for far too long.
In truth, I'd been using git for quite a while on a variety of projects, but my own
code was all still buried in a CVS archive.
I found a tool to convert the repositories, while maintaining the history,
and made the switch.
I still miss inline $Log$ and $Id$ but I'm over that.
Since switching to git, I've had a few occasions to use githooks to capture a
git push to the server.
Mostly this has been about doing simple updates or sending emails.
However, I now maintain all of my DNS configuration as a git repo.
I also have a collection of Ruby scripts which process the data (either a zone file or
a JSON file with basic zone data) and produce the master and slave Bind configurations.
Initially, these steps were decoupled, but eventually I decided to join the two so
that a push to the master branch would also force the DNS infrastructure to be
rebuilt and automatically deployed to the various name servers.
This has worked quite well, but it's not without it's difficulties.
For some of the git-hooks, I have dumped a file in a certain directory
and allowed for a cron task to notice the new file and do the appropriate
legwork.
This avoids trying to run complex processes as the git user and also avoids
delaying the git push for too long.
However, it's not particularly synchronous.
I run the cron tasks about every fifteen minutes, which means there is a lag of
fifteen minutes (worst case) from the git push until the data is deployed,
and also it means I'm running the check script 96 times a day whereas changes are
quite infrequent.
I've been thinking of either creating a Resque system and Redis interface,
so that the git-hook just submits something to the Resque job queue via a Redis
submission, or else I will code up some sort of job submission mechanism, to do
something similar.
What I'd like to be able to do in the git-hook would be to run a command such as:
This would either add an "update-dns" job to a generic queue or else it would add
some sort of timestamp or equivalent to the "update-dns" queue.
The idea would be to have something which consumed from that queue and ran the
appropriate script.
Perhaps looking for update-dns.rb or update-dns.sh in a specific directory,
and executing it.
I'm not sure I need something as advanced and complex as Resque but it seems
silly to reinvent the wheel.
I'd also like to be able to do something which was multi-host as there are times when
I'd like to do more advanced jobs other than just things called from a git hook.
I think for now, however, I'll code something up to use Redis and limit the
scope to local jobs only.
15 Jan 2016
Quite some time ago, probably like a lot of other people, I converted
my home heating system controller to an Atmel AVR circuit.
For a whole lot of legacy reasons, I haven't tended to use Arduino boards
for this.
I have a selection of Atmel ATmega8 designs for previous projects, and I
have a collection of etched PC boards which I reuse for projects such as
this.
Some day, I'll run out of etched boards and I'll switch to Arduino boards.
If for no other reason than they are surface mount unlike my own designs
which are through-hole.
Also, assembly is not required!
However that, like everything else, is a project for another day.
First off, it's worth mentioning how the system actually works, out of the box.
My heating system is the of the OFCH persuasion, or "oil-fired central heating."
For those of you not in a cold climate, this involves a boiler and burner
unit, which heats a recirculating closed water system, connected to a
collection of radiators around the house.
The burner has a blower and oil pump (as well as an electronic ignition).
This is fed via a thermostat on the boiler to make sure the recirculating water
is at a certain temperature.
Above that temperature, the burner shuts off.
There is also a pump which moves the hot water around the system and through the
radiators.
Along with this are two electrically-operated valves.
One opens the system to the radiators downstairs and one for the upstairs radiators.
Generally you never run the burner without the pump running, as this would just
heat the stagnant water inside the boiler, to no effect.
The thermostat makes sure that the water inside the boiler never gets too hot,
cutting off the burner if and when that happens.
Generally the return feed from the house will be below the cutoff so the burner
operates.
However, eventually the return feed will exceed the thermostat and the burner
will switch off, leaving the pump to circulate the water until it drops below
the low temperature threshold (hysteresis and all that).
With both electric valves off, the boiler will heat the hot water system in the
house by way of a heat exchanger in the hot water cylinder.
Generally though, I run the burner, pump and downstairs valve together.
I tend to phase in my projects rather than try to swallow the elephant in one bite.
The first phase was to remove the existing (dumb) unit and replace it with a bank
of four solid-state relays.
These
Omron
units are quite good.
They operate off a 5 volt input so they can be connected directly to the Atmel chip
(I use 5 volt Atmels - your mileage may vary).
In order to isolate the AC power from the low-voltage systems, I mounted
the four relays in a wooden box with a plexiglass front panel.
As this was enough of a project to get me started, I then took a five volt
power supply and connected it to another dumb timer.
I fed the output to three of the four relays, ignoring the upstairs system
to start with, as it isn't used very often.
At the appointed time, the dumb timer would energize, power up the 5 volt
PSU which provided an input signal to the three relays, thus turning on the
central heating system.
The phase one version of things operated for quite some time!
The second phase introduced the Atmel circuit.
I decided to simplify the programming and settled on a sixteen entry program table.
Each entry specifies a start time (HH:MM), a duration in minutes (up to 255 minutes),
a day-of-week mask, and a operation byte.
The operation was one of {H,H+D,H+U,H+D+U}{I}
where H was for hot water (just the burner and pump), D was for downstairs (burner, pump
and downstairs electric valve), U was for upstairs and I was for an electric immersion
heater, filed under TBD.
I used a serial port to talk to the Atmel board, and wired that to the house server,
temporarily.
I built a sealed lead-acid (SLA) charging circuit, and mounted the Atmel board with the 12 volt
SLA in another wooden box with a plexiglass cover.
I put a 220 volt to ~15 volt AC transformer into its own sealed box and wall-mounted that as well.
In addition, I put a separate unit with two circuit breakers on the wall.
One breaker feeds power to the Atmel transformer and one to the bank of relays.
The code keeps track of the exact time of day, and day of week.
This worked extremely well, but changes to the program schedule, or requests for a quick "burn"
of heat, were tricky.
There are times when you just want to turn on the heating system for an hour, without updating
the program.
I supported this option in the firmware, but it involved having to use /cu/ or /tip/ to
connect to the serial port and send the command.
Some time after this, I came across a
Grandstream Serial Concentrator
from Perle.
It has 24 serial ports available at the front via RJ45 sockets.
You can configure the device in a number of ways, but for this I configured it as an outgoing raw
port.
It is set up to listen on TCP port 5024 and relay data to the serial port, and vice versa.
If you happen upon one of these devices in your travels, they are well worth adding to your
equipment rack.
This allowed me to interface the Atmel board with the house network, and from there, I
developed a small web server which allows me to control the heating and adjust the program.
More on that, in a later posting.
As my main server (an
HP Proliant Microserver
naturally) runs
FreeBSD
and also is an NTP client, so it has a very accurate view of the date and time.
I have a cron task which is responsible for updating the day-of-week and time on the Atmel
board.
This takes care of two things:
Firstly, clock drift is eliminated as the time is updated every hour.
Secondly, I don't need to code for the abomination known as "daylight savings time"
as the cron task will reset the hour shortly after the clocks change.
There is a smug self-satisfaction from hearing the heating system fire to life at
exactly 0730 in the morning, knowing that the clock in the heating system had been
corrected using an Atomic Clock reference, a mere 30 minutes earlier.
The system isn't without it's challenges, and I'll cover those in a later posting.
28 Feb 2015
I hate to rain on the
Apple
parade, and in a lot of ways it's good news for Athenry,
but some of the hyperbole, particularly the political kind, is
irritating. First off, this is a data centre. We can assume land costs
are on the order of $5 million. Construction costs for shelled out DCs
are pretty low, in comparison to office buildings. The vast majority
of the $850 million budget, will be spent on servers. Rough estimates
say they might deploy up to one million of them. List prices for DC
servers run from $1,000 on up. Apple will end up paying a lot less than
that. Either way, unless Athenry makes computers, most of that money is
going elsewhere.
Intel in Leixlip could do well. Server
manufacturers around the world are no doubt already working on quotes. In
terms of staffing, the facility will need 24/7 care. If they're
quoting staff numbers in and around 100, that means four shifts of 25
DCOps (or data centre operations) staff, whose job it is to install
new equipment and replace faulty servers and hard drives. You can be
trained in how to do this in a matter of weeks. You don't need motorways
to Dublin or Shannon, because your job doesn't involve travel. These
types of facilities are called "lights out" facilities because they run
in total darkness (unless DC Ops staff are on the floor, obviously). Why
waste electricity providing lighting for banks of computers? Some "artists
impressions" of the data centres show them with large windows. Obviously
the artists in question have never looked at a photograph of a real
data centre, which can often look like a high security prison. Windows
don't help with heat efficiency or security.
The real
reason Athenry is ideal for Apple is due to the cold weather, the ESB
fibre to Dublin, along with the motorway fibre, the railway fibre, and
the imminent arrival of a new transatlantic cable, via Belmullet. To
understand the energy economics a bit better, the quoted 360MW of power
used by a DC such as this is mostly transformed to heat, by thousands upon
thousands of servers. For every kilowatt of power that a server consumes,
another 200 watts is needed to remove the heat produced. That figure can
go as high as 600 or even 800 watts. Power is one of the biggest costs
in your average data centre, and cooling is a big headache. The ratio
of total power used in comparison with power used by servers, is called
Power
Usage Effectiveness or PUE, and Apply will be gunning for a PUE in
the 1.2 range. This will mean 300MW of power for around a million servers,
and 60MW of cooling. By choosing rural locations where land costs are
low and cold winds are plentiful, you can reduce the cooling bill and
bring that all-important PUE down. It also helps if there are large
amounts of renewable energy sources around, but wind energy while great,
is unreliable. We do get calm days in the west of Ireland, and if it's hot
and calm, the PUE can start to creep up. Luckily for Apple, those days are
rare.
Wind energy is on the order of the cube of the wind
speed. Simply put, 10km/h of wind might produce 1000 units of energy,
but 50km/h will produce 125,000 units. Or 125 times the energy,
assuming that 10km/h is strong enough to rotate the blades of a wind
turbine, and 50km/h isn't too strong. While that's great, and we do
get our fair share of strong winds, looking at it from the DC perspective,
if the wind drops from 50km/h to 40km/h and those of us in exposed
areas breathe a sigh of relief, the power produced will halve. That means
having to get 50% of your power from traditional sources just due to a
drop of wind speed from 50 to 40. By the time it drops back to 10km/h,
almost all of your power needs are coming off the national grid. But let's
not quibble. Any opportunity to reduce our dependency on coal, to reduce
the amount of CO2 we produce, and to reduce the impact and need for the
unsightly Moneypoint Power Station, is a good thing, even if a lot of the
green energy we produce is being used for iTunes downloads.
In reality, Apple is known as a good employer. They've always treated
their staff well, and the one hundred DC Ops staff can look forward to
a range of benefits and perks which may make their neighbours quite
jealous. But unfortunately this isn't a windfall for Athenry in and
of itself. The people of Athenry could use it to their benefit though,
by trying to attract other players into the market. If Apple is there,
then there is a guarantee of high-bandwidth IP connectivity to the East
and to the West. In addition, wind farm developers may choose to locate
in the area, and the national grid will have to make sure that the area
is well-served by traditional power. All of this makes Athenry a good
choice for other data centre operators who to date have chosen the "T-50"
(a highly-interconnected area in West Dublin, and home to both Google
and Microsoft data centres). Good for connectivity, but not as good
for land prices and natural cooling.
All in all, it's
great news for Apple, Athenry and the west in general. But be careful
when people bandy about FDI figures of $850 million. Oh, and if you are
reading commentary about Apple in Athenry, and the article mentions the
Google and Microsoft DCs in Dublin but doesn't mention Amazon, then they
are uninformed and should not be trusted.. Amazon are the 800lb gorilla
of cloud computing, and the best-kept secret in cloud. They operate six
enormous facilities in south Dublin, which is nothing in comparison to
their facilities in US-East.
26 Feb 2015
So, last September while on board Nikea
in Greece, I decided it was high time the
Achilles 24
saw the sea. She has been languishing beside the house for a
staggering ten years (where does it go, etc). These days, I mostly seem to
sail other peoples boats, which, by the way, is a great way to sail. But
after a couple of days of pottering on board Nikea on my own, I found
that I was very taken with single-handed sailing. No, I'm not planning
on doing the Vendee Globe,
but I do like the idea of spending a few nights on the hook
in Greatmans Bay in Galway, or in Kilmurvy or even as far as Inisboffin or
Westport.
So, a plan was formed. Officially the Achilles
has been up for sale for most of those ten years, but boat buyers are
a strange and rare breed. Particularly in a recession. They come along
sporadically, and always seem to be looking for a perfect little boat,
at a knock-down price. That's something we'd all love to find. Due to
some leaky windows, hatches and other above-the-waterline entry points,
rainwater collects in the boat. For this reason, everything which isn't
bolted down (and even some things that are), were lifted out and stored
in the garage and attic. This makes it harder to sell the boat because
she looks less like a boat and more like a hollowed-out fibreglass
hull. So, slowly last year, and with more and more emphasis this year,
the Achilles is being prepared for the water. The decks were power-hosed
and theninterior was varnished. The engine is off to see the Engine Guy,
and the shopping list of new parts grows ever-longer. Who knows if I
will get as much single-handed sailing as I intend, but the process
has started well. If nothing else, she'll be in a better shape for sale
by mid-season.
I'll be keeping this blog relatively up
to date on my launch activities, and it will remain to be seen if my
optimistic launch date of April 26th actually happens.
26 Jan 2015
Unfortunately, this will probably end up as
a very long post. It will also probably end up as
being quite polemic, and may upset some people, but there
you go.
Recently, the Irish Independent talked about the
lack
of funding for Irish film. While
that is a laudable debate, unfortunately the article echoed
recent
comments by Siun Ni Rannallagh regarding additional
funding for Ardmore Studios. This, however, is not a
reasonable request. First off, the studio has been well-funded by
the Irish taxpayer throughout it's less-than-illustrious history, and
recent
developments as documented by Tom Dowling, show that
little has changed. It is no secret that the Vikings TV series
went
to Ashford
Studios. Up until that move, Morgan O'Sullivan was
a strong proponent of the studio. It is anyone's guess why
this happened. Anyway, I'm not here to bitch and moan about
Ardmore. I
wish them every success, but I would not like to see
taxpayer money poured on top of existing investments, when hospital
waiting
lists are increasing, and the rest of the economy is in
the toilet. It's not like the owners of the studio are short a
few bob. If they have grand plans for the studio, let them fund
those plans through traditional means.
I guess the
bigger (and more reactionary) concern of mine, is the overall state
of the industry. We are led to believe that the Irish film industry is
worth
tens or even hundreds of millions a year in direct and indirect
income. However, our catalogue of releases shows that the vast majority
of Irish films don't recoup their budgets, much less make a profit. A lot
of films don't even see a theatrical release, beyond a token gesture in a
handful of screens. Let me be clear, though: Irish film is as important
as it ever was. We need to see these films. We need these films to be
made. I don't dispute that for a second. What I dispute is the notion
that we have an industry. By any benchmark, it is not an industry. It
cannot sustain itself without FDI and state funding.
If we look objectively and impartially, we could see an industry
which could support perhaps ten or twenty heads of department (you may
dispute those numbers, but I doubt very much if you'd claim they were
off by an order of magnitude). In this context, I mean directors,
cinematographers, sound mixers, editors, and so on. While each of
these may also have five or six people who work for or with them,
the numbers are underwhelming. The situation is slightly better for
actors and writers, but most actors barely subsist. They do one or
two TV commercials over their careers, they do a lot of profit-share
theatre (which also has its issues), and a large number of free, student
films. Contrast this with the number of film schools in Ireland, and the
sheer volume of film students who graduate every year, fully expecting a
career in film. Like many people who run a small production company, I am
inundated with emails from students seeking unpaid intern jobs. This is
just unsustainable.
As someone who has directed a number of
short films,
and someone who threatens to shoot something long-form in the near
future, I am conflicted by the need to support people in their chosen
profession, and the absolute dearth of funding for film. Yes, the
Irish Film Board
does a good job, but rather than cultivate independent cinema, they seem
to act as a gateway to it. A sort of self-censor for Irish film. Their
short film budgets are excellent, but they only fund a handful of
films. Likewise, their feature film supports are superb, but it has
reached the stage where an Irish film can't get made without the implicit
approval of the Board. This is not good. Yes, there are films like
Charlie
Casanova, but Terry's film is the exception. Terry himself
got
sick of waiting for Film Board money and decided to go it
alone. It wasn't easy, and his tireless efforts on behalf of that film,
did a lot for it. It also helped that it is a good but controversial
film. I know of other film-makers who have also tired of shaking the
IFB money tree, and have found themselves between the proverbial rock,
and hard place.
Charlie Casanova could not have been made
without the support of a large number of people who gave of their spare
time, freely and willingly. I know Terry would have liked to have paid
them, but given the choice between not paying them and making the film,
or not making the film, he rightly chose the former. As a film-maker,
I have also relied on the "kindness of strangers" to help me make my
films. But therein lies the rub. They're not my
films. Everyone who worked on the film could claim ownership and credit
for it. I was the one who paid for the lunches, the accommodation, the
equipment hire, transport, and all of the unavoidable costs. But it is
a collaborative effort. The film doesn't stand or fall purely on the
efforts of the director or the producer. It requires everyone to bring
their best work to the endeavour. Many times, while making zero-budget
films, I have been asked (sometimes quite strongly) to make a payment
to crew members. If I could afford it, I would have no compunction
about agreeing. But there's a down side to that, too. If I had the
funding to pay for a cinematographer or an actor, I would choose the
best available for the money, seeing as money was no object. If someone
wants to start out in this insane business, and works their way into a
second-assistant camera position, they will spend a long time looking
for work as a cinematographer, while their mentors are also looking for
work. Their only hope is to work on an independent, no-budget short or
feature.
There, I've said it. I've put paid to the notion
that people just starting out in film, can earn the same as people who
have been doing it for years. They have to prove their worth, and in
film there aren't many opportunities to prove yourself. But there's
another side to this. A more important facet of the debate. It is this
cock-eyed notion that you became a film-maker to make a lot of money
(if you did, I would suggest suing your career guidance teacher). Even
at the top of your game, you can expect a life of boom and bust. Times
when the film-gods smile upon you, and you are busy. Interspersed with
times where, as one accomplished film-maker said, "I have too much
life in my work-life balance." The truth is, you decided on a career
in film, knowing only too well that your chances of making a living
exclusively from film, are quite slim. Particularly at the upper end of
the scale. As a second-second assistant director, you may be able to
keep your calendar full, between Morgan O'Sullivan and RTE, with TV3
also starting to fund production. But take, for example, RTE's most
successful independent production in many years; "Love/Hate." One
writer, and one director. If you were hoping for a gig like that to pay
the mortgage, you are in trouble.
So, if you acknowledge
that unless you're one of a very small cadre of full-time workers in
the Irish film industry, you also acknowledge that you will need to
supplement your income, elsewhere. If you don't believe me, research the
salary figures for members of the Directors Guild of America, bearing in
mind that to become a DGA member is quite a difficult process. Likewise,
look at the average annual income of SAG actors, again bearing in mind
how hard it is to join that particular union. The situation in this
country is much worse.
Where does this leave film? Having
lived and worked in Northern California, and also in London, I was
pleased to be part of a film community in both locations. A community
that worked together to make films. In some instances, such as the
London
Film-Makers Co-Op, also pooling resources
such as cameras and edit suites. I know that the
Galway
Film Centre and
Filmbase both attempt to
provide this kind of "collective" experience, but I wonder how successful
they really are at encouraging film-makers to just get out there and
make films?
What I'm proposing, is to try and set up a
similar collective, but without the offices, buildings, membership fee
or overheads. Perhaps people help you direct your first feature? But bear
in mind that you will have to drop everything and "swing boom" for their
projects, when the time comes. A collective which is open to directors,
cinematographers, actors, writers, producers, editors, animators, and
whoever else wants to make more films. If you've had enough of rejection
letters and failed funding requests, maybe this is for you? But don't
expect to be paid.
I know this will raise the ire of many,
particularly the unions, who see an endeavour like this as an opportunity
to circumvent their daily minimums. However, if even the well-funded
films aren't making a profit, where is the exploitation in decided to make
films for the sake of making films? It is quite easy to write a manifesto
and produce a contract which stipulates that any profits for the film
will be divided amongst the people involved, but that gives weight to
the lie that the film will turn a profit. If it makes people happy,
then so be it, we can let the lawyers define the terms.
The other side to this, which is rarely acknowledged, is the fact
that the only way to get better at making films, is to make films. Odd,
that. If we all sit around waiting to win the IFB lottery, if the day
ever comes, we'll find ourselves unprepared for the experience. By
filling our idle moments with the practice of making films, we become
better film-makers as a result.
My intent here isn't to
extend some sort of job-bridge scheme, or exploit students. Far from
it. The intent is to enable people to make their own films, by helping
others with theirs. The intent is to get back to the real reason why we
make films. The intent, really, is to find out if I am alone in this,
or if there is a constituency of disenfranchised film-makers, who are
itching to get out there, and make films. Is that you? Or maybe you're
repulsed by the idea of free labour? Either way, I would love to hear
your thoughts and comments. Either this is an idea which should die on
the vine, or it's the start of something.