robjam.es

How do you know when you are finished? This can be quite difficult.

You are working on something and you need to get that deliverable finished; a document, a piece of code, a project, a building or a Lego creation. But when do you now you are done?

This can be extra hard if you consider yourself to be a perfectionist!

There are a four rules that I would suggest to follow;

1. Visualize “done” from the start. Before you get started define what getting it done means. What is that end state, and ask yourself the question “could I get to ‘done’ earlier?”, “what compromises can I make to get finished sooner?”
2. Use tools to test being finished. Test Driven Development does this well. Coders are encouraged to write tests that only confirm the required functionality has been delivered. This discourages over-engineering. Once the tests pass, you’re done!
3. Assess as you go. This means asking yourself whether you are ‘really’ finished, or do you have more to do. What if you stop now, what are the negative impacts, and how much value are you providing by doing the extra work on this ‘thing’? Would there be more value moving onto the next ‘thing’?
4. Accept the 80/20 rule. Accept that it takes 20% of the effort to get to 80% of the finished product. Is it really worth spending that additional 80% of your time, to fill that final 20%?
5. Leave something to do. There is always room for improvement, but make that assessment after your product has been used, your document has been read, or your room has been lived in. You will never get it 100% right the first time round and once you start using your deliverable, you will find ways to improve.

There are of course some exceptions to the above, and you also need to question whether what you are currently working on is one of those exceptions (brain surgery comes to mind, you kind of want to get that perfect the first time!)?

But imagine how much more productive you could be if you could commit to finishing things earlier?

So hurry up and get to “done”!

Came across this really interesting infographic about customer retention. The focus is on gaming, but at a high level, the key points are critical and most often forgotten.

Click to Enlarge

Source: From the great mixpanel blog on gaming, http://blog.mixpanel.com/2011/09/16/infographic-what-you-should-know-about-visitor-retention/

The other day I talked about the concept of an Architectural deck to describe a blueprint for making decisions. And there is a great example of a well architected solution that we can test the hypothesis of the Architectural Deck against – The Human Body.

Just a reminder, there are 3 layers to our deck;

1. Foundational – The core and basis that we have to work with
2. Functional – The tried and tested solutions
3. Innovational – The things that make the solution different or special

If we try and correlate this to the human body, or more specifically a component of the human body such as the legs. This may be described in the following way;

Foundational

If we look at the human legs, there are some very specific core capabilities that they have, such as balancing upright, being able to move a person around and the materials that are used to construct them; carbon based, organic material to create skin, bones, muscles, nerves and the blood that fuels the cells. This is foundational as its core to the human legs. And the observant ones out there would have recognised “well, that’s not unique to human legs!?” And how right you are! This is a great example of the core capabilities that most animals with legs have (give or take), and so we are working with something that has been proven. The “give or take” takes us to the next layer; functional.

Functional

Human’s can walk, run, jump and do a variety of different things with their legs. But there are some tried and tested methods that have been employed. For example, there is nothing stopping you from walking backwards with you legs wide a part and feet pointing outwards (granted, it would be difficult). But the point is, we humans typically don’t walk that way. We have adopted ‘best practices’ in walking, we use our hips, bend our knees, put one foot in front of the other etc. So although there are many ways to handle movement with our legs, to get the most optimised functional movement from them, we adopt this best practice.

As an aside, think of the down stream benefits here. Pediatrists have been treating well known issues based on these best practices, how do you think they would go treating you if you always walked the way described above? Shoe’s are designed based on this premise of walking. There are many other support benefits.

Innovational

So what are the exciting innovational aspects that we can attribute to legs? Firstly, the fact that humans effectively walk upright, which would gives us the competitive advantage against certain other animals. But that is not fair, we should compare apples with apples. So think about the well oiled machine that is an sprint runner. Their muscles are finely tuned for explosive strength that is converted to speed. This is different to a marathon runner that is focussing on stamina. Sprint runner’s bones are strengthened, and the the other systems in their body (lungs, heart etc) work hand in hand to provide this competitive advantage. Although they may have the same make up, they have innovated in how they train and focus on key parts of their legs that sets them above others.

Can you think of other examples?

Enterprise Architecture is an important practice that has come out in the last 10 years. To me, its an attempt at the ‘grand unified theory‘ in business & technology. And of the most important models within Enterprise Architecture is to derive the Business Architecture which sits at the top of this tree. After discussing this with a number of people, I have got some traction to a concept I have been talking about for a while that I call it the “Architecture Deck”.

The Architecture Deck is a framework to try and conceptualise the layers of an architecture that helps make business decisions. What is great about this deck is that can quite easily be used to architect a city, a house, an enterprise, a startup, a team or a product, or absolutely anything that  can get value from an architectural blueprint.

I’m a big proponent of simple models, and this is about as simple as they get, but you will see in the next series of posts, how it is quite powerful.

Lets start today with explaining the layers (from the bottom);

Foundational

This is the lowest part of the deck, and it describes the most foundational aspects to an architecture. Its where tried and tested, and industry standards (beyond best practices) are employed. You will 99% of the times, engage this in your architectural solution, as there is no benefit to your architectural model to go outside of this. Examples would be the need to have sewerage in a city plan, or cement slab for a house, use IP networks to interconnect computers in an office environment, or use a relational database for storage in a software product. Sure there are other alternatives, but for your current purpose, there is no benefit to go outside of this. This will be described in more detail in future posts.

Functional

The functional layer describes the aspects of the architecture that have many differentiators or competitors, so you have some choice. But this is isolated for 2 special reasons; a) things here do not provide your city, business or product with a competitive advantage, & b) in immature architectures where this is not articulated, every architecture has their own version of it. This is critically important, as what we are trying to identify here is the concept of engaging best practices. Examples here could be a Rail Network in a city, the type of timber floorboards you put in a house, the order process in an enterprise, or software pattern you use in your product. As I’m sure you have already realised, there are many choices with all these examples, but the question that must be asked, “Is there any advantage in trying to be unique here, or are we best to use the methods that others have tried, tested, failed and then got right?”

Innovational

This is the most exciting layer!! This describes what makes your architecture different. What makes New York a great city that others try hard to imitate, why everyone in your street wants a house just like yours, why your office can process sales orders faster than anyone else, or what makes the iOS such a unique OS for Apple that everyone is imitating it! Its the competitive advantage, its the unique streak, and its the innovation that occurs in a city, house design, office operations or software product!

I appreciate that this is all quite esoteric, and the details will come out in future posts, but essentially the basics described here are all that you need to help you make decisions. In future posts we’ll get into the details, we will also look at some real world examples, and we’ll discuss how you can work out your own Architectural Deck.

I have been a huge fan of the Grails Searchable plugin, and its one of the main selling points when I talk to people about Grails. But something has made me go sour on it recently. A great blog post from Boris Goykhman about the plugin has prompted me to write my own.

For those of you that don’t know; Searchable plugin is a plugin wrapper in the Grails Framework for the Compass Search engine that abstracts much of the domain class to search index mappings. And Compass is an extended implementation of the robust Open Source Lucene search engine. But its the “Domain Model to Search Index” abstraction that has cause for concern.

So lets start – I think Searchable is absolutely brilliant!!! It simplifies the process of adding a search engine to a Grails application. Although apart from some straight forward indexing and searching, I don’t believe its designed to handle complex search scenarios. Sure, you can spend a lot of time bastardising your application to support many of the elements (such as searchable component and searchable reference) to try and get deep nested relationships correctly mapped, but more often than not, I find myself also bastardising my domain model to handle what I want search to do!!

I will continue to use searchable for basic type search mappings, but I have recently decided that it is more efficient to separate the concerns between domain data and search indexes. Domain models are inherently a tree structure, there are relationships, abstractions and extensions. Whereas your search index is a flattened version of this. And trying to do this flattening through decorators and mapping files is not effective.

In a recent project, I took the approach to handling the domain to search indexing data structure conversion myself. And its already started to demonstrate pay offs! I have moved to the Solr Search engine, which is another emerging implementation of the Lucene Engine. I favour it these days for its added features particularly in Geo Spatial Search capabilities.

Quite simply, my application has a SolrService that takes domain objects that I pass it and convert them to a flattened Solr Object (that I also implemented), that is then passed to the Solr Search Engine through SolrJ. All this is quite simply managed through the afterInsert, afterUpdate and afterDelete triggers in the Domain objects. But it means that I don’t have to write complex mapping files to try and map my domain structures to the search index I want to create.

What are the benefits? Most of you would state that it would take more time doing this, but I found it to be the contrary. I have spent so much time (sometimes weeks), tweaking searchable configurations and mappings to see how it would affect my index. Looking at indexes with the Luke tool to try and determine what is actually going on and then trying to perform searches against that data to see if I get back what I am after.

I have found that the search indexes are much smaller and much more performant because of it. Because you have much more control over the indexes, I create my indexes so that I don’t have to do any GORM queries at all when rendering search results. Which you can also do in searchable, but its much more complex to understand the translation.

Soon I will try and post some code on how I actually did this.

I was recently working on a project and came across this strange occurrence where the browser was quite clearly making more than one request to the server for the same document. At first I thought it was only related to when I would refresh the browser but it soon became apparent that it was for all requests.

How do you debug something like this? Well, it was a process of elimination, start removing things from the page and see when it starts to occur. Starting with the most typical, I removed js files, then css, and then finally HTML objects when the first 2 didn’t fix the problem.

Then I found it!

My application has some html rendered in the footer that is used as a template to render some JSON packets that arrive later. These html templates have img tags, but since they are templates the src=”" is not populated. As soon as I put some content in the src attribute, the double request disappeared!

And sure enough, a quick google brought this page up.

So my fix was to put a temporary img in there to stop the double requests, of which I overwrite with my JS code when I actually use the template.

BTW: The smart thing to do here would be to use an image that is on every page (like a background image). Remember it is not going to be seen anyway, and it would be much better to use something that doesn’t require another server request to speed up the page performance.

Recently I needed to manipulate some HTML for a page where the HTML did not exist in the DOM tree as yet. Or more specifically, I wanted to clone some existing HTML, manipulate it and then re-insert it to the document.

Although JQuery has a great .clone(); method, all the examples talk about cloning and re-inserting without modifying;

ie/ given the follwing HTML,

[code language="html"]
<div class="group">
<div class="user" id="1">User name</div>
</div>
[/code]

you can do this;
[code lang="js"]
$('.user').clone().appendTo('.group');
[/code]

But what if you want to do somethnig to your cloned object before re-inserting it into the DOM? Sure, with the above example, this can be quite straight forward, but other scenarios can be a little more complex.

I discovered that the best way to do thia is the following;

[code lang="js"]
var $clone = $('.user').clone()
$clone.attr('id','2')
$clone.html("Other User");
$clone.append("(User Title)");
$('.group').append($clone);
[/code]

Pretty straight forward, but the main gotcha, is how do you traverse this ‘in memory’ DOM tree? For example $clone.$(‘.someClass’) won’t work. And this is where the .find() function is your friend!

You can simply do;

[code lang="js"]
$clone.find('.someClass').doSomething();
[/code]

We recently launched Version 3 of the Posse Platform. With this release, we decided that it was time to look at our domain model. Something we had been avoiding because of the underlying fact that it meant we would need to migrate the database – a daunting task!!!

The approaches we initially looked at included;

1. using shell scripts to migrate across,
2. using groovy scripts to do the same,
3. running groovy scripts inside a grails shell, and even
4. raw SQL scripts to move from one to the other.

Our issues were that we had a domain model that came from Version 1 of the site, which was a PHP application, we then mapped that in  Version 2 of the app. Consequently we added new features that used native GORM/Hibernate, and in the end we had a small cacophony of database concepts in a single Database Model!

In the end, I decided to write a new Grails application that would handle the migration. These were the reasons to do so;

• The new Grails application created new relationships through the new Domain object definitions, the best way to keep the integrity of this was to use native GORM to handle inserts.
• I could orchestrate the steps that would need to go into the migrations by using the grails goodness available to me; webflow, controllers, services actions
• I could leverage any Helper Services that were written for Version 3 of the application to help with the migration
• If need be, there were many plugins that may have helped with things like REST, processing data etc etc
• We could do this in a repeatable fashion, we could do a series of trial deployments before doing the real thing.
• Its much more fun to write a grails app than it is SQL or Shell Scripts!!!

The process required for the migration was quite important, it dictated what steps had to happen at what point, and how to best approach it. I also made the conscious decision to work across a snapshot of the production database rather that the actual prod DB, as it was quicker (We are deployed in the cloud, and part of the migration was to move across regions). Therefore, I took a snapshot of the production database and a new database was created before being moved to the new production environment. These are the steps that I undertook;

1. Take a snapshot of prod to the migration application (pretty self explanatory)
2. Run some cleanup scripts : we identified some dirty data in our application, so we had a data cleanup party Essentially we got everyone together and got humans to determine what was a duplicate, what should be deleted, or general data cleanup (typos etc). We then used this data to cleanup the actual database
3. Next step was to migrate the tables that we could bring across verbatim. One of the constraints in the migration was that we wanted to be able to bring some of the key data across that we used in our application and retain the primary Keys. Unfortunately, GORM does not provide an easy way to do this. So I essentially wrote a series of GSQL scripts that moved tables across, keeping any information that we needed to keep, but we also had the luxury to use services in our application that help convert data. One example of this was moving currency information from a DB domain object to an Enum Class. And a Helper service did the heavy lifting of converting the data.
4. Now that most of the main data was brought across from Version 2 database to Version 3, we could use our GORM objects to create any missing information, such as relationships, cleaning up constraints or general conversion of any Version 2 data to Version 3.
5. The final step was to clean anything up that remained to be done – usually deleting data, or running any final scripts. And then migrate the newly created database to the production instance.

The magic here was in a couple of elements. Firstly, GSQL is your friend. It allowed scripts to be written to bring raw data out, which can then be created using GORM objects in the new application. Secondly was the conscious architectural decision to take any of the back end of our application (Domains and Services) and move them into a plugin. This was actually decided for altogether different reasons (as we had a couple of applications that needed to share the domain model), but it helped here as we could simply include the same domain model plugin, and the migration application would create the exact database that the production website wanted to see. Woila!!!

I initially thought that creating a grails app would be overkill for a task like this, but in hindsight, it was more of a psychological constraint thinking you are writing a whole new app to perform this function. Creating a new application is as simple as typical "grails create-app migration"

This is definitely a great approach to database migration that takes you out of the slurry of writing raw scripts, I would do it again!

I'm a huge fan of the feedly reader in my browser. But there is something that has been driving me crazy. I constantly get unread messages that are entirely unrelated to my subscriptions, and I'm a bit of a stickler for making sure my inbox is cleared – including my RSS feeds, and I am very specific to what I subscribe to.

Google Reader (and consequently Feed.ly through its support), have the Social Capabilities built into the app, which is all about showing relevant posts from people in your social network and buzz posts. But not clear way to turn this off!!

I finally found out how and its quite easy!

1. Log into your Google Reader account and go to the reader main page
2. type the following into the URL bar and hit enter: javascript:antisocial('true')
3. That's it!!!

You will no longer get posts from Buzz or feeds that you haven't subscribed to!

Sometimes you need to change the Session Timeout in your grails application. But the important thing to know is that its not really an attribute your application supports, but its something that you can tell your container to change.

To do this, you need access to your web.xml file, which is not available by default. First run;

grails install-templates

Then you need to edit "src/templates/war/web.xml" and add or change after servlet-mapping;

<session-config>

   <session-timeout>60</session-timeout>

</session-config>

The value that you pass in the session-timeout is in minutes. And you're done!