Website Hosting Upgrade

Andrew Hallam | | 22 March 2008, 07:17

This website was previously hosted on a shared FreeBSD server at TextDrive (now Joyent). It served us well for several years, but suffered from typical shared server problems, like needing reboots and pauses while waiting for shared resources.

Yesterday I migrated the website to a shiny new Shared Accelerator server, running OpenSolaris. It appears to be responding noticeably faster.

If you notice any issues I’d appreciate it if you could let me know.

Putting Yourself Out of Work

Andrew Hallam | | 27 April 2007, 07:39

It is too common in the software industry to see contractors and consultants try to make themselves indispensible. The usual approach is to hang onto some key knowledge. I take the opposite view. Working to put yourself out of work is the best strategy.

As a consultant, who also implements solutions, I’ve always tried to make myself despensible by:

  • Sharing knowledge freely. I always gain more knowledge from a project than I can ever give away. That makes me more valuable, without having to keep things to myself.
  • Doing my best to ensure that the client can run with the solution without me being there to baby sit it. Having a bus factor of one is bad for any organisation.

When you put the client first there is a good chance that the same client will offer you another project. It is also a good incentive to stay sharp and do a good job. Taking any other approach just seems wrong.

Sure, I have my own preferences for technologies and how things should be done, influenced by my experience. That’s part of why clients engage me, but my preferences should not dominate the project. The focus should always be to provide sustained business value for the client. You cannot deliver that if you are the only person who has the knowledge.

[Hat tip to Gerald Weinberg.]

Note to self: Remember to always follow own advice.

Feeling Lucky

Andrew Hallam | | 25 March 2007, 01:00

Today my wife and I got to vote for a new state government. None of the major parties in the election inspire any confidence, but hey, at least we actually get to vote for a democratically elected government.

After voting we took the kids to the beach for a swim. The water was cool and clear, and some of the waves were surfable. Everyone had fun.

In the grand scheme of things, life is pretty damn good. There are many many others who are far less fortunate.

Job Titles

Andrew Hallam | | 27 February 2007, 18:18

Personally, I dislike job titles. If they are too generic they are meaningless, and if they are specific you get pidgeon-holed. However, yesterday a client suggested an informal job title that sums up quite well what I’ve been doing lately:

  “Remover of Mystery”

I like it! I’ve been asking a lot of who, what, where, why and how questions lately. I’m also in the process of sifting through vendor marketing blurb looking at how several classes of products might actually meet my client’s requirements.

Lost - Google Mojo

Andrew Hallam | | 9 February 2007, 05:02

I can’t seem to get the results I want when searching Google. I’ve lost my Google mojo! Previously it was just a matter of a few tweaks to keywords and I was on the scent of the information I wanted. Not this week. It’s been slim pickings.

Perhaps it’s the fact that the topics I’m searching for contain more generic keywords, or the topic is more specific. e.g. “SQL Server disk utili(s|z)ation statistics”. Not sure. It’s not Google’s fault, but it is frustrating when a search engine that has served you so well for so long is not providing the results you have come to expect.

Cultural Sensitivity

Andrew Hallam | | 25 December 2006, 21:25

David Megginson says it’s OK to wish him Merry Christmas, and says he’s tired of people searching for culturally sensitive things to say. In the interest of culturally sensitivity I thought I’d pass on a message received from a friend, minus the culturally insensitive first paragraph.

Greetings to Everyone at this time of the generally accepted calendar.

I wanted to send some sort of Christmas greeting to my friends and colleagues, but it is so difficult in today’s world to know exactly what to say without offending someone. So I met with my attorney yesterday, and on his advice I wish to say the following:

Please accept with no obligation, implied or implicit, my best wishes for an environmentally conscious, socially responsible, low stress, non addictive, gender neutral celebration of the summer solstice holiday, practiced with the most enjoyable traditions of religious persuasion or secular practices of your choice with respect for the religious/secular persuasions and/or traditions of others, or their choice not to practice religious or secular traditions at all.

I also wish you a fiscally successful, personally fulfilling and medically uncomplicated recognition of the onset of the generally accepted calendar year 2007, but not without due respect for the calendars of choice of other cultures whose contributions to society have helped make our country great (not to imply that Australia is necessarily greater than any other country) and without regard to the race, creed, colour, age, physical ability, religious faith or sexual preference of the wishers.

By accepting this greeting, you are accepting these terms:

This greeting is subject to clarification or withdrawal. It is freely transferable with no alteration to the original greeting. It implies no promise by the wisher to actually implement any of the wishes for her -/ himself or others and is void where prohibited by law, and is revocable at the sole discretion of the wisher. This wish is warranted to perform as expected within the usual application of good tidings for a period of one year or until the issuance of a subsequent holiday greeting, whichever comes first, and warranty is limited to replacement of this wish or issuance of a new wish at the sole discretion of the wisher.

Thanks Mark. :-)

Christmas Break Has Arrived

Andrew Hallam | | 24 December 2006, 01:54

Wow, it’s here already! If you are taking time off, have a great Christmas break. And, have a healthy, happy, and prosperous…

Happy new year

May the year not pass too fast, and you have time to stop and smell the roses.

Information Overload Death Spiral

Andrew Hallam | | 19 December 2006, 16:51

Arrgh! Inbox out of control. Hundreds of unread blog posts. Almost three thousand unread mailing list messages. Two blogs to write. Books to read. Supposed to be working on other interesting projects, and having a life. Doing a lot, but getting nothing done. Something has got to give.

Time for a break, and a rethink on how to successfully survive in the information age. Maybe it’s time to just go surfing. Yeah, that will fix it.

Publishing Event Details Using Atom

Andrew Hallam | | 17 December 2006, 01:24

Many government agencies have a need to advise other agencies, and the general public, of natural and man-made events. Those events might be weather warnings, agricultural emergencies, traffic incidents, bushfire updates, or many other things. Those events all occur at a location, so there is also a spatial component.

Bushfire

Notification of an event, or an update on an ongoing event, is news to someone who is interested in that event. Imagine a web of network connections between multiple agencies, created using common Web technologies, and across those connections flow news items encoded in a simple machine readable format.

Each information source in the web is a “news feed”. Effectively, by publishing a news feed an agency allows it to be “syndicated”. Other agencies can republish news item via their internal systems, and integrate the data into their workflows.

The most commonly used format for news feeds is RSS, and the latest standard in that area is the Atom Syndication Format. Add in some GeoRSS to provide spatial information, and you have an ideal combination to transfer event details between agencies.

Example Scenario

Your national meteorological agency publishes severe weather warnings on their website, via fax, etc. Other agencies, who have operational responsibilities, would like to obtain those severe weather warnings in a machine readable format. That enables them use software applications, and business rules, to process all items in a news feed and take whatever action is required.

Thunder storm

For instance, if a severe weather warning were to be issued for a certain location then the appropriate staff could be notified via email or SMS. It could also be published on an internal website, or in an a corporate system, with hyperlinks to any supporting information.

To make this sort of automated processing possible each news item needs:

  • Date and time information. i.e. When it was issued, updated, etc.
  • Categories so that software applications can decide what is of interest, and to whom.
  • The ability to contain arbitrary HTML content, and a mechanism to support custom extensions.
  • Links to other related resources.
  • A locational reference that is understood by all parties receiving the news feed.

The Atom Syndication Format takes care of the first four. GeoRSS takes care of the fifth. Atom’s category mechanism can also be used to provide a locational reference using named locations or regions (as shown in the example below).

Example Atom Document

Severe weather warnings from the Australian Bureau of Meteorology have been used to create this example because the BoM make a lot of their stuff available on the web. This allowed the inclusion a lot of links to related information in the Atom document (which makes it a more RESTful design).

Download example document (it’s a bit too large to include inline). Open in any text or XML aware editor.

Some good advice is to not create your own custom XML language unless you really have to. You can do a lot with the Atom Syndication Format, and it’s associated Atom Publishing Protocol.

One of the most powerful features of the Atom Syndication Format is its “link” element. The above example uses it to link to a map image that displays the location of the Severe Thunderstorm Warning area. There could also be links to any other supporting resources.

The “category” element can also handle multiple single level categorisation schemes per entry. In this example: warning status, warning type and the affected weather forecast districts.

Try Atom and GeoRSS. You’ll like it!

[tags]atom, georss[/tags]

Backups - Why

Andrew Hallam | | 4 December 2006, 02:54

Digital data has become a part of life for a lot of people. Some of that data records moments in time that can never be repeated. Some is an artefact of business. The reason why we do backups is to reduce the risk of losing that data, because:

  • We have assigned some personal value to it, or
  • The loss of that data would have a cost (money or time).

Everyone knows that they should do backups, but as my dentist says. “You should only floss the teeth that you want to keep.” Does it happen? Usually not until they have lost something important.

Hard disk on fire

Now, before going any further I have to follow a time honoured tradition, when writing about backups, by asking: “You are doing backups, right?”

This question is actually a little dangerous. You can answer “yes”, feel good, and still be quite exposed to data loss. A better way to look at the situation is to ask yourself:

  • How could data loss occur, in my situation?
  • What are the realistic risks of that data loss happening?
  • How valuable is the data?
  • What is the cost of data loss?

The answers to those questions depend on your personal situation. However, the answers will give you a much better starting point for working out a backup strategy that works for you.

Data Loss and Risk

I can think of a few ways to lose data:

  1. Hardware failure,
  2. Malicious acts,
  3. Natural and man-made disasters, and
  4. Human error

Before going any further, have a think about which of those you have considered to be the highest risk. Make a mental note. Done? OK, let’s keep going.

Hardware Failure

Most people I talk to think that backups are done to protect themselves from hardware failure. Hard disks do suffer from mechanical and electronic failures. A hard disk failure is a catastrophic event, and all data is usually lost.

Hardware failure

However, most people also overestimate the risk of hardware failure, and don’t consider other risks. Think of it as similar to how humans overestimate the risk of dying in an aeroplane crash. It does happen, and all on board usually perish, but the chance of dying in a car accident is significantly higher.

Overestimating the risk of hardware failure can actually cause people to underestimate the risk of data loss. That sounds totally counter-intuitive, but bear with me. If a person thinks that their biggest risk of data loss is from hardware failure, but they haven’t ever suffered such an event, their perceived total risk of data loss will be low. Throw in a bit of “it won’t happen to me” and backups might get done, occasionally…maybe.

Malicous Acts

Malicious act

People steal things. People write destructive viruses. People intentionally delete other people’s data. It happens.

Your backup strategy needs to consider these threats and weigh up the risks. If you back up your precious family photos to an external disk drive you have two copies (good). If someone steals both your computer and external disk drive, you are…well…screwed (that’s bad).

If a virus starts deleting files, and can get to the external disk drive because you have it mapped as a drive letter, its the same story. Storing two copies of data on separate devices is probably not good enough.

Good anti-virus protection, a firewall and physical security are important parts of an overall data protection plan, but they will not be discussed further in this post.

Disasters

Natural disaster

Disasters happen, at different scales. If your house burns down due to an electrical fault that is a disaster for you. If hundreds of houses get burnt down in a bushfire that’s a disaster for the larger community.

Take your pick:

  • Fire
  • Flood
  • Lightning strike
  • Severe storm
  • Power surge
  • and more

All can wipe out your computer hardware, and possibly your offline storage media.

How many times have you seen images like the one above on TV? People sifting through the ruins of their home looking for family photos and heirlooms, sobbing “It’s all gone”. The risks might be low, but it happens.

Human Error

Humans do silly things. We drop things that don’t bounce well. We delete stuff we need later. We overwrite later versions of files, etc.

Human error

Something like 80% of all data loss in corporate environments is due to human error. I’d wager that the ratio is similar in the home and small office, but probably more due to lack of familiarity with the technology than anything. Home users aren’t usually IT professionals, and nor should they need to be to prevent data loss.

When was the last time you accidentally deleted a file that you needed, or wiped out some changes that you had made? I did it two days ago. In a rush to clean up my desktop I put a bunch of files in the rubbish bin. I then emptied it without checking what I was deleting. I lost a file I had been working on. (I had a backup so I only lost the changes I’d made that day.)

Reducing Your Exposure

The first step is to recognise all the risks to your data and make a realistic assessment of them. All the above can happen, but when formulating a backup strategy you should recognise that:

  1. You are probably more of a risk to your data than any other factor.
  2. Other risks also exist, and should not be ignored.

(What did you think was the highest risk to your data?)

The next step in creating a backup strategy is to consider the value of your data. (That will be the next instalment in this series.)

[Note: The images used in this post were obtained from iStockphoto and are subject to their licence agreement, not the Creative Commons licence used for this website.]

« Previous |

Powered by Textpattern | Tranquility White made TXP-ready by Textpattern Templates