Connascence and primitives

In response to http://blog.8thlight.com/colin-jones/2012/06/05/on-obsessions-primitive-and-otherwise.html

"Connascence between two software elements A and B means either: 

1. that you can postulate some change that would require B. to be changed (or at least carefully checked) in order to preserve overall correctness, or

2. that you can postulate some change that would require both A and B to be changed together in order to preserve overall correctness."

I think it’s fair to say that the reason we organize primitives and operations on them in larger structures, encapsulate them in modules or as separate entities, is to make sense of them - as humans. We need some way of having them meaningfully catalogued, so we can wrap our brains around the concepts of a domain. We require to continuously enable ourselves to understand what the program does by growing and constantly refactoring the design. We need to make it easy to identify the parts of it that require alteration when the business finds a program behavior necessary to change. And again, we do all of this only because we as humans need it, because this is how our brains work, how we think, learn and reason.

The computers could hardly care whether we introduce the domain concepts to the program design. If it weren’t for us, the programs could as well be still written in Assembler, or even in the machine code. As much as the previous sentence seem blatantly obvious, it should help me make my point - by having connascence as a tool we can more easily make the conscious decision of using either the primitives or the domain objects. I truly believe that the most important factor for choosing either of the solutions, is to help us to minimize the number of the interconnections required by a potential change (ie. make the “total connascence” lower, and better localized).

Example:

Where there is no (or minimal) ambiguity what a “price” means, and how to operate on it, it may be safe to just let it be “BigDecimal”. But then, if the application grows larger and larger, it may so happen that the other concepts will start appearing, such as “taxedPrice”, or “weight”, etc. Then it may become more difficult to perform a change related to “price”, because many entities are connascent, ie. “BigDecimal” and operations on “BigDecimal” mean different things in different contexts. Consequently a programmer may use incorrect methods on wrong data, change parts of code which are semantically unrelated, or do not change these parts that are related.

All in all based on the definition of connascence in order make a “perfect modification” in a program I want to:

• carefully check as little of it as possible,
• be as certain as possible that I checked enough,
• change as little of my program as possible,
• be as certain as possible that I changed enough.

The list above may not be exhaustive, and I confess to actually creating it a bit ad hoc, but it can be extended as needed based on a definition of connascence. IMHO all that matters is that we keep connascence localized (ie. easy for a human reader to spot), and minimize the number of connascent software elements. That of course as a rather generic statement surely may be applied not only to the problem of primitive obsession, but to a large number of design decisions.

A serious take on Data-Rich Development - Part 2

In the last part I explained how I managed to prepare the histogram of class changes across issues. This was certainly not everything that you can do with the data that I generated. The next step was to get more detailed information on each class. I decided that for sure I want to see how number of modifications changed over time and to achieve this I prepared a chart you can see below; it shows how many issues “changed” this class at a specific moment in time (month being the resolution)¹:

Note that in this case I decided to use standard line chart since the information that I wanted to present is not really that complex – I certainly don’t want the “cool charts” to become my golden hammer. It is also worth noting that after using this data for some time I decided to normalize Y-axis, so I can compare metric on different classes at a glance. See for instance second example below, where you can clearly see that over time number of both modifications and defects decreased – it’s not marked as sharply as on stock market, but if you see a number of these charts (one above being an example), the existence of a trend becomes obvious.

It’s still hard to say whether this trend means good or bad – it is obvious that this class doesn’t get changed, but it still can mean that certain problems exist, eg. the class got too big and rarely any modification is justifiable by a business reason.  To nail down the cause, you’d need other metrics to provide you more context. I’ve been already thinking of possible extensions that could exploit more sophisticated ways of visualization, when for example one could be able to see how size of a class changes on the same timeline. Theoretically it would be awesome to see coverage there as well but this is unfortunately not possible in our case – if you however can do it, go for it, I’d be thrilled to see that. And if I were to choose how to visualize that I’d probably go for a chart a’la GapMinder which sort of out of the box would enable having combined view of metrics for many classes at the same time. Anyhow, I’m not sure which information would benefit you the most, but it’s very much worth exploring :)

I was not planning to explore this problem in an orderly fashion of any sort, because when I research a topic I like to do a bit of jumping from one thing to another which helps me get a better grasp on all aspects of the particular problem). I decided that for a next step I want to go more into the correlations of classes across issues (I again got inspired my Michael Feathers – http://michaelfeathers.typepad.com/michael_feathers_blog/2011/09/temporal-correlation-of-class-changes.html). The first visualization that got created out of this concept was a graph of all possible correlations (ie. classes that get changed together as a part of the same issue) above certain threshold for whole project and it looked like that:

The size of a node (representing a class) is proportional to the aggregate number of issues when a class was changed together with other class, and correlation is depicted as a link between the two. This visualization certainly looks cool and also is interactive – you can pan the area, zoom, move nodes around… actually look foryourself on Protovis site. What’s the downside then? There’s just too much information – it does show you an overview of areas having a strong coupling (see yellow), it will highlight boundaries of application modules (see blue), but it’s almost impossible to get more specific information out of it. So it’s good as a start but you need a next step here, something that would let you dig into the details why the situations is as it is and whether you should do something about it.

I’m planning on describing ways of resolving this in Part 3 so for now let me just show you another way of visualizing the same information. What I’m going to present is IMHO much more useful when you need to focus on the correlations (especially identify where they don’t make any sense) rather than classes (correlations lower than 3 were filtered out):

The concept is quite similar to a previous one: nodes are classes, links are correlations. Then, around the whole circle classes are positioned in a specific sort order – by package name. Having them in such an order let’s you apply a simple heuristic – whenever there’s a link between a two remote locations of a circle there is potential unnecessary coupling between two separate packages… and while there may be a relationship in the code it’s at least suspicious if these classes get changed together too often (change frequency is represented by color, increasing from green to red). On the other hand even if the correlations are close (in the same package), but there are lots of it, it still can have negative meaning – for example the package may be too large. I didn’t play much with this visualization so there may be many other ways of analyzing and getting valuable information out of it. Moreover there’s an amazing tool for doing much morepowerful visualization of this kind, and as soon as I learn how to use it, I’ll write more on its potential.

In the next part… right, I’m not gonna lie to you, I have absolutely no clue about the next part, besides that there’s going to be one. Maybe I’m going to get more into the detail how I decided to present information for a single class… or maybe I’ll describe possible use cases to you can employ these charts for… or something totally different. Not sure – stay tuned.

---

1.  If you happen to have the length and scope of different issues wide-spreaded this metric will count all changes within single issue as one and result in overestimating importance of “quick fixes” and underestimating “long enhancements”. Because of that recently I modified this metric not to count all modifications in a single issue as one, but instead do it per-day basis. Then if a file is modified many times on different days, the number of days when it’s modified is the number we’re looking for.

A serious take on Data-Rich Development - Part 1

“It's about taking the data that we have at hand in our development work and really using it.  If we are making a decision about whether to refactor a piece of code, we should be able to see its churn and complexity trends, and tie them back to events that happened over time, and the actual features which triggered the work.  Right now, it seems that we often look at our decisions through the pinhole of the present, ignoring what we can learn from our code's past.”

Michael Feathers, Data-Rich Development

A few weeks ago I had a very fruitful conversation with my colleague Bogdan Lachendro from the team at Sabre Poland. Yet again we started discussing Technical Debt and I can assure you it is not an easy topic if you consider project of our magnitude (15 years of development, millions line of code). The crux being that you want the best way there can be to identify code spots you need to fix, as your time is limited and the potential areas to clean up is almost infinite.

What to do... what to do…

You might think – boy-scout rule!... and yeah sure that’ll help, but on a humongous codebases it got a nasty habit of working rather like a shotgun – sometimes you fix the code that really needed it, and for the most of the time you don’t… continuing with a metaphor, if you shoot long enough you might get lucky and hit the target more often than not. So it’s fine to do it, yet IMHO it helps to keep the entropy from increasing, but it’s not really as helpful as you would want it to be.

At this point I can hear you screaming – code metrics bro’, code metrics! PMD, Sonar, FindBugs! Ok, cool – they’re fun to use, but if you’ve ever worked on a project of that size you know how that ends up. You get sooo many warnings that you don’t even know where to start. Again – cool stuff, but without extra cues, it won’t fly.

What we really needed, was a method that’d tell us how we can get the best bang for the buck. We needed to have something that’d help us deliver faster, easier, have less bugs in the software (preferably in the places that customers use most often). Yes I do mean all of it when I’m saying “reduce technical debt”, for the very reason that technical debt is not really so much “technical” as the name would suggest. At this point we already knew that we needed a whole truck of fortune cookies to tell us what that freakin’ business would want 10 years on from now and where our clients would find most bugs, so we can go ahead and clean it up… or, maybe, just maybe we could extrapolate from historical data. After all we are able to extract data for over 6 years of development and there should be enough information to get meaningful conclusion out of it.

And what we came up with was to use the information from issue tracking system¹ (we use ClearQuest) and the source code (we used ClearCase and are currently on SVN) and consolidate it. That way we would knew which areas of the code were changed the most and could predict that they might be the main targets of modification in upcoming months (I’m using months, ‘cause I’m not really sure if much more time got left – because of that thing with Mayans and their calendar...). Sounds familiar, right? Yeah, I won’t lie to you; we were under strong influence of what Michael Feathers is preaching about for some time now (eg. Getting Empirical about Refactoring).

The hacking started and after not that long a time I managed to get my data out of ClearQuest (thank you CQPerl!) and combine both SVN and Clear Case information to get list of tuples in this format: [ISSUE_NUMBER, CHECKIN_DATE, FILENAME]. The next step was to visualize it somehow... actually I was pretty skeptic about using yet another line/bar/pie chart and craved for a picture which meaning just gets to you. In fact I strongly believe that in a today’s world it is more important than ever that you not only pick the right information from the data, but also choose a right way to present that, and with the right tools this can actually be pretty easy to achieve. After a moment of googling I came onto Protovis – really amazing visualization library that let you do very powerful graphics almost out-of-the-box. And after not more than 15 minutes the graph below was brought the world:

Histogram depicting number of defect fixes across which classes were changed (bubbles represent files, the size of bubble is directly proportional to number of defect fixes file was modified in)

I was not entirely honest with you… A few more little details about what I did:

• filtered out files that does not appear in more than N issues,
• made size of a bubble a quadratic function of number of issues that a file was modified in (ie. size_of_bubble = number_of_issues ^2),
• grouped colors of bubbles by a package (or it might have also been a part of code from an original example, I don’t remember)
• I don’t want to share too many details about my project, but trust me these are really the places in the code that you would expect to appear on this graph (and now we have data to defend to our position which is pretty cool).

I myself had two most obvious conclusions after looking at this “rich histogram”:

• There is around 15-20 files that stand out (if you consider we have tens thousands of files in a VCS, this is pretty strong) – you probably start your technical debt discussion from this classes, ie. make them smaller, clean up the implementation, increase coverage,
• files in an “orange” package are changed more often than other – this needs more analysis, on whether the package is that large, or they are often changed all together… and then maybe because of coupling

What next you can do with this graph? Possibilities are plenty… things that I did already or am planning to in a near future is to let user:

• let user see histogram either on class level or on package level
• filter out specific modules or packages
• filter out test code
• let user calculate the histogram only for selected date range

Now this graph alone does not tell you yet whether these are the files that needs your care, but it is one step forward from being completely ignorant about the historical context of changes in your codebase and its potential impact on the business in the future.

In the next posts I will continue to explore various ways you can use this information to exploit your data to the limits.

---

1) We might have also considered to pull the information VCS, but we used ClearCase for a long time and only recently switched to Subversion - and since on ClearCase you don’t really have a concept of a commit that was a blocker.

Hibernate lifecycle

Because I'm very busy lately I'll keep it short. Having been annoyed by Hibernate, all the magic happening under the hood and rarely any information on what is really going on I decided to do a little research on hibernate lifecycle. I don't think any document of that sort exists on the web so I decided to share it with you. I tried my best to double check everything but I might have omitted something. If you spot an error - please let me know.

See Hibernate lifecycle document

Design patterns in Scala: Adapter

One might think that when considering Adapter pattern the ol' good Java implementation is 'as good as it gets'. But as you're probably already guessing I believe it is not. Surprisingly (ye.. right) Scala shines also on this front.

Say the code we're dealing with is:

class Communism { def makePropaganda = ... }

class Capitalism { def usePublicRelations = ... }

If you wanted to adapt 'Communism' to behave as 'Capitalism' (say we need it during visits of important foreign ministers ;)) in Java typically you'd do it like:

class CapitalismAdapter(val communism : Communism) extends Capitalism {
 def usePublicRelations = communism.makePropaganda();
}

// Usage:
new CapitalismAdapter(new Communism).usePublicRelations

If you're in control of 'Capitalism' class it'd be even better to do 'Capitalism' a trait or create 'ICapitalism' trait which'd be implemented by 'Capitalism' class.

Anywho.. Let's do it Scala way.

implicit def systemTransformation(communism : Communism) : Capitalism = new Capitalism {
def usePublicRelations = communism.makePropaganda();
}

// Usage:
new Communism.usePublicRelations

That may seem like a one simple and insignificant improvement - but in fact it's pretty powerful stuff. After all an explicit conversion you did in Java is nothing but noise (by now you most probably noticed that Scala is the noise-annihilator).
Also it saves you from all that extra keystrokes and postpone RSI ;).

And with more difficulty it gets only better. Consider following problem:

For reasons unknown (Architectus Reloadus asked us to do so) we want to extend some lists in our system with a 'fancyMethod'. These conversion should apply only to the lists that contain elements able to be converted to a 'String'.
To do it in Scala you implement an implicit conversion of a 'listToStringList' and put a requirement on a user of providing an implicit conversion of each element to 'String'.

trait StringList { val list : List[String]; def fancyMethod = () }

// implement implicit conversion of 'List[A]' to 'StringList'
implicit def listToStringList[A](aList : List[A])
// require to provide (another) implicit conversion method (A => String) in a scope of this conversion usage
(implicit aToString : A => String) = {
  new StringList { val list = aList.map(aToString) }
}

// Usage:
def foo(s : StringList) = ()

// should cause error:
foo(List(1, 2, 3))

implicit def intToString(i : Int) = i.toString

// and now it works:
foo(List(1, 2, 3))

I won't provide the Java solution for this problem just because I care about my sinews..

An important thing to remember is that implicit conversions in Scala are not transitive. It is still possible to do such conversion (with calling any method explicitly) but it must be 'requested' with ':' operator.

class A; class B; class C; class D

implicit def aToB(a : A) = new B
implicit def bToC(b : B) = new C
implicit def cToD(c : C) = new D

// Usage:
val a = new A
println((a:B):C) // A => C
println(((a:B):C):D) // A => D

Design patterns in Scala: Command

Many people say that Command pattern in languages of functional paradigm is not really a pattern but actually an inherent feature of a language itself. I don't know about other languages but in Scala it is not entirely true. While it is rather quite easy to come up with Command pattern implementation - you write a closure and it's done ;) - but variety of possibilities that language gives you is pretty amazing.

Let's start from the very beginning.
I believe that the three following points represent the most important aspects to consider when choosing appropriate implementation of Command Pattern in Scala.

• what the signature of "Command" used as a parameter looks like
• how do you construct a "Command" instance
• how do you assign a "Command" instance to a value

The most straightforward solution to Command pattern implementation is:

// Example 1
def command(i : Int) = println(i);

// Usage:
// 1
def invoker(cmd : Int => Unit) = cmd(1) 
// 2
invoker(command _)
// or
invoker(command)
// 3
val savedCommand = command _

However concise and elegant it has important shortcomings. Firstly you get very wide and crude interface - (Int) => (Unit) means nothing after all. You won't communicate a lot of information with that and a user can pass any command to an "invoker" method (including ShutDownReactor procedure ;]). The second thing is a nasty wildcard you have to use when assigning an instance of a command to a variable. (which you may optionally use in 2. - and if you don't compiler will do it for you).

You can have a bit cleaner solution with following piece of code.

// Example 1
def command = (i : Int) => println(i);

// Usage:
// 1
def invoker(cmd : Int => Unit) = cmd(1) 
// 2 a) prepared command
invoker(command)
// 2 b) ad hoc command
invoker((i : Int) => println(i))
// 3
val savedCommand = command

The method presented above is actually the partially applied function of a function from the previous example. You gain the advantage of not writing the '_' (placeholder) when assigning an instance to a variable - what IMHO both simplifies and makes it a little less magical :). And if you wonder what would happen if you've added it anyway, the answer is simple - you'd get a partially applied partially applied function ;P with following signature:

() => (Int) => Unit

To solve the issue of wide interface you might use the more classical example of a Command pattern with a bit of functional language coolness on the top ;)

// Example 3
trait Command { def apply(i : Int) }
object DefaultCommand extends Command { def apply(i : Int) = println(i) }

// Usage:
// 1
def invoker(cmd : Command) = cmd(1) 
// 2
invoker(DefaultCommand)
// 2 b) ad hoc command construction
invoker(new Command { def apply(i : Int) = println(i) })
// 3
val savedCommand : Command = DefaultCommand

Pretty nice - it gives you very clean nice interface with possibility of extending it with whatever responsibility you like (what was obviously missing in first two examples). Nothing comes for free though - the ad hoc construction of a command class is more verbose than a simple command function.

If you're cra^M^M^Mbrave enough you might even try something like this:

class CrazyCommand extends (Int => Unit) { def apply(i : Int) = println(i) }

And now the nice thing is that you may both use it as in 3. example and also pass it to a function defined in previous code snippets. Though I have no idea why anyone would like to do that ;)

While I'm at it I'd remind you about a cool Scala feature that let you extend class behaviour at instantiation and thus you can do the same with:

new Command with (Int => Unit) { def apply(i: Int) = println(i) }

Fun Exercise! ;)

type CoolCommand = (Int => Unit) with (String => Unit)

def coolFunction(cmd : CoolCommand) { cmd(1); cmd("one") }

The implementation of a class that may be passed as a parameter to a 'coolFunction' is left to a reader and may be impossible :P

Design patterns in Scala: Singleton

I am always keen on improving my knowledge of Scala. Second thing is that I really enjoy to share what I learned with everyone (I do really hope people eventually gain much more interest in Scala :D). That's why I decided to make series on .. GoF design patterns in Scala.

For starters I've chosen our old friend - Singleton. Let's go then!

object Singleton

// Usage:
val singleton = Singleton

Done! Thx for reading..

..just kidding, let's dig a little deeper into that.

You are probably curious about thread safety of this solution. The actual code generated in bytecode and decompiled with JAD looks like that:

public final class Singleton$
    implements ScalaObject
{

    public Singleton$()
    {
    }

    public int $tag()
        throws RemoteException
    {
        return scala.ScalaObject.class.$tag(this);
    }

    public static final Singleton$ MODULE$ = this;

    static 
    {
        new Singleton$();
    }
}

There are 3 things to consider:

• Singleton object is initialized in static block so thread safety is guaranteed by JVM


• Every time you reference Singleton object in your code, Scala compiler translates this call into 'Singleton$.MODULE$',


• This implemention has also quite cool semantics of access modifiers (which is a bit more advanced topic; search for 'companion module' to learn more).

An important thing to notice is that Singleton is eagerly initialized so if you try:

java.lang.Class.forName("Singleton$")

it'll cause initialization to be performed. For now I have no idea how to force 'object' in Scala to be lazy initialized. The only solution I came up with would be:

// Incorrect (see the bottom of article)
class LazySingleton private () {
  def apply() = this
}

// Correct
class LazySingleton private () {
}

object LazySingleton {
  lazy val INSTANCE = new LazySingleton();
  def apply() = INSTANCE
}

// Usage:
val singleton = Singleton()

The trick is to make default constructor private and have apply method of 'object' class type return the only instance of LazySingleton class.

Disadvantage of second implementation is a small but noticable difference in usage. First - requires '()' [apply[ operator to be used. Second - it returns different type of class depending on whether '()' is used (class LazySingleton) or not (object LazySingleton):

scala> C()
res1: C = C@304648

scala> C
res2: C.type = C$@1b9e7fc

---

Modified 2009-06-23: Previous implemention of lazy singleton was incorrect and led to wrong conclusions.

Wroclaw Area Situated Scala Enthusiasts group established !!

I am happy to announce that Przemysław Pokrywka and Piotr Adamski established WrASSE (Wroclaw Area Situated Scala Enthusiasts) here in Wroclaw!! It's really amazing to meet people that care about Scala and hope for this great language to become the next Java :D

If you're curious how this image is connected to this post visit WrASSE discussion group

There are regular meetings planned in the (hopefully near) future, but a group is veeery young and all your great ideas for WraSSE are welcome!

If you are Scala fun (slash zealot .. whatever ;)) and do care about your professional future (which would be Scala mostly ;P) I encourage you to join this group and actively participate!

My two cents on Agile adoption

Agile is like teenage sex, everybody is talking about it, most are not doing it and those that are doing it, are doing it wrong.

paraphrase of James O. Coplien ¹

So a guy - let's call him John - wants his company to be hip'n'cool and be Agile. He goes by the book, put some things into work and yeeeeaahh.. nooo.. - people scream, tools don't work, tests breaks, client's pissed.. He begins ranting on Agile being unable to deliver him its expected awesomeness.. trying harder and harder.. right until it breaks completely.

Yeeah...

You know Albert Einstein was a pretty smarty guy? He actually said:

Insanity: doing the same thing over and over again and expecting different results.

That is exaaactly what John is doing. I am honestly amazed how vast majority of people approach this problem. It may be characterized as: "Doesn't work? Let's ditch it". I mean.. WTF? Are they really that lazy to make any alterations? It's either that or they thought they'd be anointed by Holy Ghost with Agile skills.. I mean be reasonable you don't expect to become Software Architect one month after graduation...

However there's still a question to answer: "What might be a better solution than moving back to RUP" (which is bad and stinks and even IBM knows it :P).

You know how TDD cycle looks like don't you:

What if I've shown you something quite similar:

First - move slowly, take small steps. Otherwise you won't know what works and what doesn't, you will overwhelm your team² with too much change and slow down development process down to nothing.

Second - use feedback! You're getting negative - may it be - in TDD red tests have the same value as green ones.

Third - don't expect to become Agile star overnight.

It requires time.
It requires effort.
It may deliver profits.
If you're not determined to take a risk don't put blame on a process. Just admit it.

---

1. From his superb presentation on DCI.

2. "Gently introduce change to a team. Don't expect things to suddenly change overnight." Ryan XXX [don't know last name]

Been busy preparing presentation "Scala for practitioners"

Last month it was pretty silent on this blog and I got to say sorry for that. It was quite busy month for me and I couldn't even find the time to tweet not mentioning blogging. The most important thing that came out to live during this time was a presentation - on probably the best language in the world - titled: "Scala for practitioners". I gave it for the first time yesterday in Power Media S.A. (which is the company I'm currently cooperating with as a freelancer :)) and you can find the project I used during the presention here on github. The primary goal was to show Scala as language having real business value here and now, being really close to get significant impact in software projects. From all the feedback that I've gathered I guess I can safely state - it was a success! :D Next I'm planning on giving this presentation on Wroclaw JUG. It actually needs some minor corrections (what again will probably cause absence of new posts on this blog .. ;P) but I really hope it will trigger more enthusiasm for Scala in Java community. I'm also thinking on doing some screencasts on Scala but the exact idea is not fully matured yet, so all tips are welcome!

For all of you not to feel like you've lost another 2 minutes of your life reading this post:
Mindblowing example of how cool is regexp matching in Scala (if you think you've seen it try checking it out anyways - it's not the same old regexp matching example :P).

Delivering the code is like doing the laundry

I hate doing the laundry.. it's definitely in the top 5 of housework activities I hate.. Recently I talked with my friend about - no, not about doing the laundry ;) but nonetheless the topic came around. He was actually amazed how might I not like this particular activity - 'cause in his eyes it was nothing more than putting your dirty stuff to the washing machine, switching it on and doing nothing for an hour or so. I couldn't answer right away so I decided to focus next time I'm doing the laundry on what exactly I don't like about it. Identifying the precise reason behind reluctance to a certain thing or activity is after all the first step to overcome it ;)). Then the day came - stuff started to pour out of the clothes basket (as usual) and I was forced to clean this mess up.. To spare you the details, the list of activities being part of doing the laundry is:

1. sorting the laundry
   
2. stuffing it to the washing machine
   
3. going to the market because you forgot you were low on the @#$#@ washing powder
   
4. taking the laundry out and hanging it on a dryer
   
5. collecting the laundry after it dries
   
6. ironing (personally at this point I'm too pissed about the whole activity and skip this point)
   
7. folding clothes and putting them to the drawers, wardrobe or wherever they should be stuffed..

So this basic task is as a matter of fact damn strategic operation ;] And that's the precise reason why I hate doing the laundry :) (And will probably continue to not like it :]) At this point you must already be wondering "what the hell is this guy prattling about?" Let's begin a second story then! ;)

Recently I had a small discussion with my colleague about the (un) importance of testing. He defended the concept of testing being time-consuming and thus not acceptable in software business. He represented the old school of "yes I know it's important but we don't have time for this stuff we need to get the implementation fast". What drew my attention was the last part of this sentence "we need to get the implementation fast". As if in the software business we were responsible to push the code out of the door and not give a damn where/when/what it'll be used (for). This is not the first time I hear this opinion - software developers too often forget that the code is not what we produce (yes, I know it's easy to see it this way). Our job is however do deliver functionality to the client and not the crappy, low-quality kind of functionality which she/he never wanted. The high-quality and precise reflection of her/his requirements (which she/he might have not even been aware of.. that's the hard part).
When unit testing is not the elementary part of software development the whole activity of delivering a single requirement consists of:

1. requirements elicitation
   
2. coding
   
3. QA testing
   
4. fixing the code
   
5. QA testing
   
6. deployment
   
7. client finding bugs (this is the optimistic version where the software actually reflects her/his requirements)
   
8. fixing the code
   
9. deployment

Finally what's the resemblance between doing the laundry and delivering the code, you ask. There is a clear tendency for both to be interpreted too narrow! :)

Just like doing the laundry is more than putting clothes to the washing machine,
delivering the code is more than coding the implementation.

It's not about how fast you code the requirements but how soon you get them right

The actual software is just a byproduct of the process of building an understanding of a given domain.
Jonas Bandi

The misconception behind the RAD tools comes from the desire to implement the software (ie. do the coding) as quickly as possible. People believe that producing software is hard - and they're correct, it really is! The problem is that it is not the implementation that is tough. Meeting blurry, vague wishes of your client - sometimes erroneously called requirements - is the hardest part in software development. And unless you work on really dumb system for feeding the data using few hundreds of forms (which most probably means you do software for govmt) you really need to get the domain right.

On all projects with non-trivial business domain you spend most of the time analyzing clients requirements, evolving (expanding and refactoring) the domain model to meet your clients demands. You spend hours in front of the whiteboard with the marker in your hand talking about the domain. You do Proof-of-Concept, you revert, do it again, revert.. And then if you're totally, 100% sure what's really going on in this damn business domain (and BTW most probably you're still wrong about it ;)) you go to do some front-end, mailing, logging and all that supportive stuff.

In my first job I had a teammate who was a decent programmer but he shone big time. I spent a gigantic amount of time on learning every detail in technology we used and I knew the class model very good. He knew just enough about it but still when it came to talking about the project he always knew better. The thing was it that he focused on and understanding specifics of business domain what almost always gave him massive advantage.

Most of the time you do the knowledge crunching.

OO design, reuse and serendipity

Steve Vinoski said pretty wise stuff about reusability of REST-based systems. He said that powerful stuff as Unix small languages was probably incidental just because the design (unix pipe) was based on modularity and simplicity.

It is entirely the same for Object Oriented design. Keep your interfaces simple (ie. make them have clear boundaries [PDF warning!]) and your abstractions simple (ie. having clear and sharp definitions [PDF warning!]* - and modularity comes for free. Simplicity enables modularity!

The important part is that, what is easily forgotten, reuse is not only about using parts of the code on different projects - this should actually be quite rare - I mean most of the time you use external frameworks to do generic stuff.. don't you?. But you should constantly reuse domain classes by using them second, third and nth time in different contexts. And by different contexts I mean implementation of the most exotic requirements of your clients. It's definitely not a good sign if you catch yourself saying "naahh.. we can't use that method for the new requirement, 'cause it's doing this thing and the other thing to do that stuff here.. here.. and over theeeere.. we gotta write a new one".

What you might rather try is:
- refactor (decouple),
- test,
- reuse.

Do it KISS, don't strive for too much generality, but do it right! After the umpteen requirement (and refactoring) you'll be amazed that you have evolved a very sharp abstraction with clear boundaries. Moreover you gained a fair amount of domain knowledge because you kept on building larger coherent structure and not only solving single problems. Always remember about GPS** (Gain Perspective Stupid).

"Serendipitous reuse" is definitely a good sign and if you do OO right it'll come for free.

* The rule of a thumb is that if you have to use two metaphors (or more, sic!) to explain a role of an abstraction to the person from outside the project it means that your domain model stinks in that particular area.
** Coined by my good colleague and great software developer Piotr Wójcicki.

My view on different level of abstractions in OO

Lately I've been trying to invent my very own classification of classes of abstractions you come across when creating object-oriented software. As every more-or-less formal classification it should give me (at least that's what I hope for) a possibility to infer various qualities and evaluate use-cases for each of the class of abstraction. It also gives you quite a nice foundation to introduce heuristics and conception of refactorings from/to certain class of abstraction.

I've come up with the idea of 3 categories which I call "nth class citizens":
- 1st class citizens - (abstract) domain classes,
- 2nd class citizens - (abstract) classes included in general purpose libraries - various data structures (eg. Collection, Set, Date, etc.),
- 3rd class citizens - primitive types and classes (eg. int, Long, BigDecimal).

In case of 1st class citizens the main problem is the metaphor. Every abstraction is described by either an abstract or real entity constituting its role. It is not always possible to model a goal and responsiblities of an entity in a programming language of your choice. It may happen that an object lose some meaning or demonstrates additional features. The first issue is easy to resolve as it requires us only to judge whether the lost qualities were necessary to fully express the intent of object existence. As for the latter:
- demonstrated features (qualities) may be the actual functions of the modeled abstraction - in this case by incident we get the useful extra "feedback of the metaphor",
- the new features have nothing in common with the modeled abstraction - the domain model is contaminated with false information. These features cause unnecessary confusion and draw away the attention of developers from the actual intent of choosing certain abstraction.

2nd class citizens use a general interface to communicate (this interface have a precise definition in the domain of computer science). Incidentally it matches the one of a modeled business domain. In most of cases (as in 80/20 rule) there is no need to hide the information they share with the external world - you should just use the standard "wide" general purpose interface. There is however a danger that:
- there are in fact 1st class citizens in disguise (eg. Set<Map<Account, Order>>),
- they either share too much information or their messages are too generic and hard too understand by their collaborators. In both situations they should be rather represented by 1st class citizens (implemented with encapsulation).

3rd class citizens, well... they just exist :). In most cases you just push them around - and in 99,999% of situations you should not use them to perform any business operation (eg. DON'T DO chargeClient(BigDecimal price)). They should always be "represented" by 1st class citizens (encapsulation).

This article is one of the first drafts and I hope I'll come back to this topic. There is never enough of polishing object oriented modelling, design and implementation skills! :)

the beauty of scala traits and self types

So I'm working on a small throw-away project to check out the Scala+Jersey+XStream stack and I've come across a nice implementation pattern (more precisely I found a way to use one I've heard of previously in the slight different context, see James Coplien presentation on DCI and the article on the subject).

I hold a reference to an instance of XStream class somewhere in my app. It is responsible for marshalling objects and it looks like this (examples are shortened for clarity):

val XML = new XStream(new xml.JDomDriver()) {{
    setMode(XStream.NO_REFERENCES);
    alias("courses", ScalaCollectionClasses.ListBuffer)
    alias("course", classOf[resources.Course])
}}

That looks nice - the initialisation block gives the instantiation of XStream object nice DSLish look of a spec. But when I decided to use both XML and JSON and following chunk of code appeared on the screen.

val XML = new XStream(new xml.JDomDriver()) {{
    setMode(XStream.NO_REFERENCES);
    alias("courses", ScalaCollectionClasses.ListBuffer)
    alias("course", classOf[resources.Course])
}}
val JSON = new XStream(new json.JsonHierarchicalStreamDriver()) {{
    setMode(XStream.NO_REFERENCES);
    alias("courses", ScalaCollectionClasses.ListBuffer)
    alias("course", classOf[resources.Course])
}}

Ok.. duplication strikes you right away. So how do you fight that monster? There are actually a few options to choose from:

• inheritance
  

  
  class MyXStream extends XStream {
      setMode(XStream.NO_REFERENCES);
      alias("courses", ScalaCollectionClasses.ListBuffer)
      alias("course", classOf[resources.Course])
  }
  val XML = new MyXStream(new xml.JDomDriver())
  val JSON = new MyXStream(new json.JsonHierarchicalStreamDriver())
  

  
  


• delegation
  

  
  val XML = new XStream(new xml.JDomDriver()) {{ configure(this) }}
  val JSON = new XStream(new json.JsonHierarchicalStreamDriver()) {{ configure(this) }}
  def configure(xstream : XStream) = {
    setMode(XStream.NO_REFERENCES);
    alias("courses", ScalaCollectionClasses.ListBuffer)
    alias("course", classOf[resources.Course])
  }
  

  
  or
  

  
  val XML = configure(new XStream(new xml.JDomDriver()))
  val JSON = configure(new XStream(new json.JsonHierarchicalStreamDriver()))
  def configure(xstream : XStream) = {
    xstream.setMode(XStream.NO_REFERENCES);
    xstream.alias("courses", ScalaCollectionClasses.ListBuffer)
    xstream.alias("course", classOf[resources.Course])
    xstream
  }
  

  
  


• implicit conversion
  

  
  val XML = new XStream(new xml.JDomDriver()).configure()
  val JSON = new XStream(new json.JsonHierarchicalStreamDriver()).configure()
  def xstreamToConfigurableXStream(xstream : XStream) = new {
    def configure() = {
      setMode(XStream.NO_REFERENCES);
      alias("courses", ScalaCollectionClasses.ListBuffer)
      alias("course", classOf[resources.Course])
    }
  }
  

Ok as for the first one - inheritance sux.. I mean really.. inheritance is for most of the time the last resort and you know it.

Second and the third one are ok.. I mean most of the time I'd say ok to this solution and implement something like this in Java.

Fourth one.. imho this is a hack and clear abuse of implicit conversion in Scala. Personally I believe that the use of implicit conversion can be justified when trying to bend the existing API for your purposes (ie. trying to seamlessly extend non-functional Java classes to full-blown Scala versions).

So what is the best solution I've found?

  trait Configuration
  /* 1. */ trait DefaultConfiguration extends Configuration { /* 3. */ this: XStream => 
    /* 4. begin */
    setMode(XStream.NO_REFERENCES);
    alias("courses", ScalaCollectionClasses.ListBuffer)
    alias("course", classOf[resources.Course])
    /* 4. end */
  }

  /* 2. */val XML = new XStream(new xml.JDomDriver()) with DefaultConfiguration;  
  val JSON = new XStream(new json.JsonHierarchicalStreamDriver()) with DefaultConfiguration;  

1. Create a trait that will bind the reference to an instance of a class it extends to "this"


2. Create an instance XStream for XML marshalling and extend with 'DefaultConfiguration' trait which..


3. in fact will bind the instance of XStream to 'this' and..


4. perform an initialisation of a bound object

A powerful (and strikingly easy after you see it for the first time and get the 'aha moment') mechanism for introducing behavior into existing classes. It gives you a perfect decoupling and let you introduce an explicit abstraction for the extension ('configuration' in this case). And now I can do:

  def foo(xstream : XStream with Configuration) = "thanks for configured xstream instance bro"

Which basically means: "give me an xstream instance.. but the configured one if you may!"

I am not sure I have fully explained the reason why I like the last one the most.. I got the gut-feeling this is the right path to go. Reusable, expressive, explicit.

Keywords vs API, continued

Przekonany postem Jeffa Atwooda (którego mimo wszystko jeszcze czytam :P) postanowiłem, że decyzją pragmatyczną będzie zmiana języka bloga na 'informatyczne esperanto' czyli angielski. Z dniem dzisiejszym wszystkie posty będą ukazywały się w tymże języku.
-- Translation --
Jeff Atwood (whose blog I continued to read in spite of everything :P) convinced me in his latest post that it'd be a pragmatic decision to switch the langugage of posts to 'Esperanto of software world', ie. English. Therefore starting today all posts are written in English.



Here we go then!

In one of the older posts I mentioned that it may be quite a problem to migrate from Scala to Java world. The keywords chosen for Scala might have been used already for name of the methods or classes in existing Java APIs (e.g. 'match' or 'yield') thus preventing us from using them in our code. BTW the well known example of such a case is Java evolution itself (1.4 -> 1.5 and the appearance of 'enum' keyword')

It appears that in Scala nothing is impossible! Well it is not feasible to define/use functions as presented below:

def match() = "Hello world! I am a 'match'!"
def yield() = "Hello world! I am a 'yield'!"

match()
yield()

Yet nothing prevents us from doing it this way:

def `match`() = "Hello world! I am a 'match'!"
def `yield`() = "Hello world! I am a 'yield'!"

`match`()
`yield`()

It is incredibly useful not only for Scala adoption among Java developers but also quite helpful during the development of 'Scala greenfield projects'. While in Java it is not that big issue (due to relatively small number of keywords) in Scala it'd have probably started being a bit of pain in the ass (with keywords like 'val' or 'match').

The conclusion? Scala rox! :D

404 done the right way

Ponieważ po raz kolejny napotykam się na beznadziejny komunikat 404 .. postanowiłem poprawić jeden z nich. Na poprawce spędziłem +/- 30 sekund i przepraszam za ewentualne błędy.

Przed Pawłem:

Po Pawle:

Po Pawle wersja minimalistyczna:


Dlaczego uważam, że moje wersje są lepsze? Pozwolę sobie skomentować niektóre fragmenty komunikatu:
1) "Welcome to 404 error page"
Przeciętnego użytkownika Internetu doprawdy niewiele obchodzi czy kod błędu w protokole HTTP to 404, Alfa8 czy Dupa16.. Chciałbym, żeby programiści wreszcie zaczęli sobie z tego zdawać sprawę. Z drugiej strony świadomy użytkownik internetu zdaje sobię sprawę że nie istniejąca strona to kod 404 - więc po co mu o tym przypominać?! Kolejna rzecz, że obsługa błędów powinna być transparentna dla użytkownika i wtedy nie będzie musiał się uczyć różnicy między 404 a dajmy na to 410. Informacje czytelne dla komputerów zostawmy komputerom.

2) "Welcome to this customized error page. You've reached this page because you've clicked on a link that does not exist."
Ok.. odniosłem wrażenie jakbym celowo otworzył tę stronę i to co czytam było początkiem artykułu o kodzie błędu 404. Naprawdę nie wystarczyło krótkie i treściwe "Page does not exist" ?

3) "This is probably our fault..."
Nie miejsce i czas na unoszenie się honorem. Stare porzekadło ludowe mówi: "Klient ma zawsze rację". Komunikat powinen brzmieć: "This is totally our fault! We beg for forgiveness!"

4) "but instead of showing you the basic '404 Error' page that is confusing and doesn't really explain anything, we've created this page to explain what went wrong."
I co? Mam być wdzięczny, że straciłem 10 cennych sekund mojego życia czytając komunikat błędu ?!?!

5) "You can either (a) click on the 'back' button in your browser and try to navigate through our site in a different direction, or (b) click on the following link to go to homepage."

Chociaż mogli a i b pogrubić bo zajęło mi z dobre kilka sekund (a to dużo; odsyłam do badań z usability) żeby się dowiedzieć co mogę zrobić.

Reasumując:
Tworząc komunikaty z błędami kierujmy się zasadą KISS. Zaoszczędzimy użytkownikom dużo cennego czasu, który mogą spędzić, np. stymulując polski rynek browarniczny ;)

Scala nabiera prędkości

Widać, że zainteresowanie Scalą rośnie (Hura!). Chyba czas najwyższy poważnie zacząć w nią inwestować (np. ruszyć Dage'a :P).

Książki o Scali które są - lub w najbliższej przyszłości będą - wydane.

Ponieważ już wiele osób mnie pytało jak zacząć przygodę ze Scalą, poniżej zamieszczam kilka przydatnych linków:

Prezentacja o Scali - Konkretna, pragmatyczna, krótka (108 slajdów :P).

Tutorial - szybkie wyprowadzenie do języka. (15 stron)

Przegląd możliwości języka - naprawdę "mind-blowing". Polecam!! (kiedyś 55 stron w PDF - teraz dostępne na stronie)

Scala poprzez przykłady - wprowadzenie do języka. (145 stron)

Specyfikacja języka - tylko jeśli wybrałeś czerwoną pigułkę ;) (180 stron)

Streszczenie: Investigating architecture and design

Jedną z rad w przeczytanej ostatnio przeze mnie książce "Pragmatic Thinking & Learning: Refactor your wetware" autorstwa Andy'ego Hunta jest prowadzenie wiki. W takiej wiki może znaleźć się wszystko od ciekawych cytatów, poprzez przemyślenia aż do streszczeń książek oraz artykułów (itd. itp.). Streszczanie książek/artykułów, podkreślanie ważnych fragmentów, dopisywanie komentarzy etc. samo w sobie jest polecane przez Hunta jako świetny sposób na lepsze przyswojenie tematu... Nie będę jednak przepisywał porad z książki Andy'ego bo nie o tym miało być :) (choć gorąco zachęcam do jej przeczytania!).

Zamiast tego zamieszczam streszczenie artykułu Neal'a Forda "Investigating architecture and design" przeplatane moimi przemyśleniami.

Jeśli komuś spośród garstki osób, które odwiedza mojego bloga ;) podoba się pomysł takiego właśnie dzielenia się wiedzą będę od czasu do czasu publikował notki tego typu. W przeciwnym razie, żeby się nie pozbawiać i tak nielicznego grona czytelników :P odpuszczę sobie.
Czekam na feedback!

---

Notatka powstała przy użyciu pluginu do VIM'a - viki oraz konwertera deplate (dla zainteresowanych wystarczy: sudo gem install deplate).

---

Źródło:
Evolutionary architecture and emergent design: Investigating architecture and design[1]


1 Introduction
==============

Architecture is being separated into two categories:

* application architecture - coarse-grained pieces that compose an
application - which is further dived into:
+ framework-level architecture - the combination of frameworks,
+ application architecture - logical separation of concerns.
* enterprise architecture - how enterprise as a whole consumes
application.

+++ Metaphor for software architectures

Enterprise architecture => city planning

Application architecture => building architecture


2 Definitions of architecture
=============================

"In most successful software projects, the expert developers working on
that project have a shared understanding of the design system design.
This shared understanding is called 'architecture'. This understanding
includes how the system is divided into components and how the
components interact through interfaces." Ralph Johnson

"Architecture is about the important stuff. Whatever it is." Martin
Fowler

"Stuff that's hard to change later." Neal Ford


3 Important note
================

!!! Architecture -> Evolves !!!

Because it consists of elements that must exist before you start
building an application.

!!! Design -> Emerge !!!

Because it is grown over time.


4 Useful equations
==================

"Flexible architecture" + "Reversible decisions" = "Evolving
architecture"

"Just-in-time dirty hacks" = "Technical debt" = "Rising entropy"


5 Complexity
============

* essential,

!!! Watch out for essential complexity! It might be possible to
!!! lower it by asking customer to consider her/his choices more
!!! carefully. Money argument is the ultimate way of doing that.
* accidental.

Accidential complexity:

1. Just-in-time hacks.
2. Duplication.
3. Irreversibility.

+++ Because genericness adds entropy, you damage your ability to evolve
the design in interesting ways early in the projects.

[1] http://www.ibm.com/developerworks/java/library/j-eaed1/index.html

moje książki (for share)

Ponieważ kupowanie książek na Amazon w ostatnim czasie stało się wyjątkowo drogie (albo raczej realistycznie mówiąc ceny powróciły do dawnych standardów) postanowiłem, że być może lepiej zorganizować coś swoim w środowisku. I tak doszedłem do wniosku, że najłatwiej będzie się po prostu powymieniać :D Posiadane przeze mnie książki znaleźć można tutaj.