Phillipus – Page 11 – dada beatnik

Archi – an Open Source ArchiMate tool (part 3)

PhillipusSeptember 20, 2013October 21, 2017

(Continued from Part 2)

Release early, release often

A very early version of Archi was ready by mid-March 2010. I demoed this to a roomful of enthusiastic delegates at an Enterprise Architecture Practitioner Group (EAPG) meeting in Birmingham, UK. It was warmly received. The features were very basic, but there was enough there to whet the appetite.

An alpha version of Archi, version 0.7.0, was released on April 8^th 2010. Even though an alpha release, this version actually implemented most of the core functionality. All three ArchiMate layers were implemented, as was support for multiple Views and you could save the diagrams as image files. Further alpha and beta releases were made in April and May 2010. These versions focussed on fleshing out the feature set, bug fixing and getting early user feedback.

The official public version 1.0, was released on June 18^th 2010 in time for the summer EAPG meeting. This version implemented the main feature set together with all the “hints” available in the Hints window, and Help documentation. Again, it was received well. We also had a website, a Google forum, and a small, but growing, Archi user group. The 6 months were up. (I should add, not to promote my ego, that this was all done by myself – there was and has only been one developer/documenter/website maintainer/helpdesk/designer/stunt coder.)

Beyond 6 months and Global uptake

It turned out that Archi was not just being used by the Jisc projects in UK higher education. Professional and student Enterprise Architects everywhere were downloading it. As a result of its early success, 6 months funding turned into 24 months funding, taking me up to December 2012. I have to extend enormous gratitude to Jisc for providing me with the funding to work on Archi for all of that time.

During that overall period I added many more features in response to user feedback posted to the Archi User Forum and received by email. This feedback resulted in Archi’s infamous “Magic Connector” and inspired many other features that contributed to its ease of use and popularity.

Feedback was generally positive. Ideas came flooding in from users around the world. News of Archi was spreading fast. Here are some examples of feedback from the first few months of its release (anonymised):

“…first impressions are that it’s a much nicer experience to actually model with… The feeling I get from Archi is that it’s helping me to create shapes, link and position them rather than jumping around dictating how I can work with it. And the models look much nicer too… I think Archi will allow people to investigate EA modelling cost free to see whether it works for them, something that’s not possible at the moment.” – Staffs University, UK.

“It’s a really impressive application and very nicely built on top of Eclipse.” – Healthcare Institute in the Netherlands.

“Archi – nice work – looks very promising – I hope you can find the time and money to keep building it out!” – EA consultant, Australia.

“I’m new to EA world, but Archi 1.1 makes me feel like at home! So easy to use and so exciting…” – Senior Engineer, Russia.

“The most interesting tool to date for Archimate is yours, also because it is based on Eclipse and GEF.” – PhD student from France adding extensions in the Telecom domain.

“Version 1.3 looks great! We are rolling Archi out to all our architects next week. The ones who have tried it so far all love it.” – Senior Architect, Insurance company, US.

“It’s a nice alternative… In my first attempts of using the tool I found it to be very useful and promising.” – IT Architect, Delft University of Technology.

I think it’s interesting to note the efficacy of a fast feedback loop between me, as the developer, and an enthusiastic group of users. As the musician Robert Fripp likes to say, it’s important to remain “small, mobile, and intelligent”. Without the constraints of formal processes and methodologies imposed by the parent organisation, I was able to develop the software iteratively and quickly, implementing new features that the users, and I, felt were important. Open beta testing allowed me to fine tune the features as the users suggested.

Archi allows for EA conversations

It was clear then, and still is now, that the availability of Archi as a free tool allowed for a greater degree of sharing and discussion of EA in the UK Higher Education sector, and, indeed in the wider community, both commercially and non-commercially. Archi was the core tool used at various Modelling Bashes at UK Universities, a London King’s College ArchiMate workshop, and many of the discussions in the EA Practice Group meetings revealed that practitioners were sharing models created using Archi.

As I attended more EA Practice Group meetings, overheard discussions, and saw increasing numbers of presentations using screen-shots of Archi models, I saw Archi’s presence growing more and more. Why was this?

That Archi is free and open source is one good reason. Another reason is that it runs on all three main platforms – Mac OS X, Windows, and Linux. But looking at the rich feature-set of some of the major modelling tools tools out there I wonder why anyone would choose Archi. Could it be the pricing model? The bigger tools are great if you’re a large company and can afford the licence, but not so great for the single user, the EA student, the small enterprise, the cash-strapped University.

Archi’s popularity puts a great pressure on me, as the only developer. Users want Archi to support some of the features that these other tools have. For free, of course. They would love to have model repository support, sharing of models, versioning of models, export to this format, import from that format, and so on. Of course, there’s always the get-out clause – “It’s Open Source. Perhaps you’d like to contribute…?”

But whatever the future for Archi, what has been clear to me is that in its existence it has provided very many people with a means to share models and EA conversations that might not otherwise have been possible.

(Go to Part 4)

Archi – an Open Source ArchiMate tool (part 2)

PhillipusSeptember 18, 2013January 4, 2015

(Continued from Part 1)

Beginnings

As I wrote in Part 1, I had to get something out of the door within the given 6 months of project time funded for Archi. We had said, in the first planned stages, that the deliverable would be a “proof of concept” application, with the hope that we might get additional funding beyond the 6 months if there was enough interest in the project. I was never really interested in creating a “proof of concept” tool, I wanted to deliver a full-blown application, ready for real-world usage.

Some developers are, let’s say, a little unsure of themselves, inclined to conceal their code behind a veil of false modesty. I’m happy to say that I’ve never suffered from this affliction, as I like to get the first few lines of code out there as soon as possible. If it’s junk, I want to know. Maybe I’m naive, or overly enthusiastic, or maybe it’s because, as the character of Mister Wolf in the film Pulp Fiction says, “I solve problems”.

If I’m writing a desktop application, such as Archi, I like to start straight away by creating an empty application window with the App’s title in the top bar, an icon, a basic menu and an “About” dialog box declaring me as the author (to take the blame, not the credit). That’s it. This only takes a few hours to put together, but it means that you have an App. It doesn’t do anything except open, close and show the “About” box, but you have a freakin’ App. It’s like creating a frame for a picture, or an outline for a novel. On day 1 you can launch your App and you can show it to people. It’s a tangible thing. This is really important if you want to encourage interest in your endeavour, and convince people that you just might be able to do the impossible. Also, this is Open Source software funded by public money, so transparency is key.

After the initial code was written, I created a project at SourceForge and committed the code. It was a CVS repository then, now no longer visible since the transition to Git a year later.

Frameworks

Choosing a modelling and graphical framework was the next task. At the time, I could already hack something in Eclipse’s Graphical Eclipse Framework (GEF) which would take care of the boxes and lines, and I could use the Eclipse Modelling Framework (EMF) for the model, persistence, and model notifications. Additionally, there is the Graphical Modelling Framework (GMF) which combines the two frameworks but this turned out to generate too much Voodoo Code.

modelling wtf — The Eclipse Modelling World

Voodoo Code is a term I use for frameworks that generate code and artifacts based on a given set of user-provided meta-models and input by some mysterious and opaque process. Something very, very clever is going on in that Voodoo, but you sure as hell don’t know what it is. Many lines of code are automagically generated, but you don’t know how to modify them, or what might break if you do. I have to confess to being horribly confused by GMF, I feel scarily not in control; and of course there’s the old problem of the lack of documentation in the Eclipse eco-system (don’t get me started on that subject). And I only had 6 months to work with. It was like this Dilbert cartoon:

I went with the “old school” option – a combination of EMF to create the model and persistence framework, and GEF to handle the drawing graphics, and I would hand code the “glue” between the two frameworks. The cross-platform, Java-based, Eclipse framework could handle all the windows, menus and other GUI requirements. At least this way I could understand and maintain the code without feeling that it was all sinking away from under my feet. I should say that the EMF does in fact generate code, but it’s boiler-plate code, easily modifiable and very easy to update (and EMF has great documentation and support). Eclipse did a great job with that framework. Not so sure about GMF, though. A colleague once remarked that he wished GMF could be “un-invented”.

I spent the next two months developing a basic EMF meta-model for the ArchiMate model and the Application model, and linking this to the GEF to draw the boxes and lines, and battling with the Eclipse framework in order to beat it all into some coherent shape. At one painful point in the development process I grew doubtful, abandoned EMF completely, and spent the next three days and nights coding an alternative. On the fourth day I realised that I had effectively re-written EMF, although in a half-baked manner. The following day, I put it all back again, and really appreciated how well EMF has been devised. I wasn’t using Git version control in the early development phase, I was using CVS. If I had been using Git, I could have created any number of alternative, experimental branches with ease.

Go to Part 3

Archi – an Open Source ArchiMate tool (part 1)

PhillipusSeptember 17, 2013October 21, 2017

Archi is a free, open source, and cross-platform desktop application for creating models using the ArchiMate modelling language that I developed between 2010 and now.

ArchiMate

What is ArchiMate? I suppose you might say that it’s a bit like UML or BPMN, except that it addresses a higher level of business modelling abstraction, the domain that interests so-called “Enterprise Architects”. For me, I like to simplify this and say that ArchiMate is just more “boxes and lines”, albeit with a special meaning.

I suspect that ArchiMate and its associated framework, TOGAF, are not widely known outside of the arcane world of Enterprise Architecture and its cohort, and so the tools that its practitioners employ are rather specialist. Consequently, these tools tend to be somewhat expensive as they are aimed at mid to larger organisations, and sold using the “enterprise” level of subscription model. This is fine for those organisations that can afford it (banks, insurance companies, manufacturers), but can prohibit the individual user or student who may just want to learn ArchiMate, or at least experiment with it.

Background

In 2009 I was working for an organisation within an organisation within an organisation (perhaps a dis-organisation?). It turned out at the time that some UK universities had started to take their first steps into this brave new world of Enterprise Architecture and ArchiMate as part of a Jisc funded initiative aimed at improving services and processes in the UK higher education sector. For many participants, this was an entirely new domain, and committing to a particular proprietary toolset to support the project was considered to be risky. Some universities had already bought a single licence for one of the existing ArchiMate tools, whilst others had started to use Microsoft’s Visio drawing tool. For larger businesses the cost of these tools and support might be no big deal, but for cash-strapped UK universities it was a serious concern, especially as they were only doing this as part of a special programme and didn’t want to incur any recurring costs if it didn’t work out. The absence of a standard export format also raised concerns of the possible danger of proprietary lock-in.

Jisc proposed that additional funding could pay for the development of an open source ArchiMate “proof-of-concept” modelling tool that could be used by those institutions as a “taster” of ArchiMate before committing to a full-blown solution, and also provide a means to make an intervention in promoting an open exchange format for ArchiMate. The proposal was for:

…the creation of an OSS tool oriented towards those making their first steps with Archimate and with functionality somewhere between template-driven drawing tools like Visio and “enterprise class” tools which would fill a space in the market and provide a platform for implementing a standard export.

Given my previous development experience with Eclipse and some familiarity with Eclipse’s modelling framework (EMF) I was given 6 months to develop such a basic ArchiMate modelling tool.

I like a challenge.

6 months for one developer to create an open source, cross-platform tool for modelling an unknown (to me, at the time) visual language. I’m not sure if you’d consider this ample provision or not. I’ll just say that, in retrospect, 10-12 hours a day of coding for 6 months carries health risks.

I set to work in early 2010, reading up on the ArchiMate specification (dry and boring when read with nothing concrete to hang it on) as best as I could given that I had never been involved with Enterprise Architecture. A first draft plan was agreed, and implementation would be basic given the limited time-scale.

Go to Part 2

We are all Journalists

PhillipusSeptember 13, 2013July 12, 2014

It seems that the US Senate wants to redefine the term “journalist” with the so-called “Shield Law”.

http://blogs.wsj.com/washwire/2013/09/12/senate-panel-approves-media-shield-law/

Of course, this means excluding Julian Assange as a “journalist”:

http://www.theatlanticwire.com/politics/2013/09/terrorists-and-julian-assange-arent-considered-journalists-senate/69355/

http://www.huffingtonpost.com/2013/09/12/media-shield-law_n_3912955.html?utm_hp_ref=politics&ir=Politics

Whatever they say, with the power of the internet at our disposal, we are all journalists.

Xcode and Asynchronous Unit Testing

PhillipusSeptember 12, 2013January 4, 2015

Unit Test support is greatly improved in Xcode 5, with new features such as the Test Navigator and individual test runs. XCTest is the new framework, replacing OCUnit. I’d hoped there would be better support for testing asynchronous calls with all of this new stuff, but sadly this is not the case. So, what’s the issue here?

Some methods in the iOS API work asynchronously, on a background queue, and are completed on a completion block, at some point after invoking them. Here’s an example:

- (void)testSaveAndCreateDocument {
    NSURL *url = ...; // URL to file
    UIManagedDocument *document = [[UIManagedDocument alloc] initWithFileURL:url];

    // Call the asynchronous method with completion block
    [document saveToURL:document.fileURL
        forSaveOperation:UIDocumentSaveForCreating completionHandler:^(BOOL success) {
            STAssertTrue(success, @"Should have been success!");
        }];
}

In this Unit Test, we’re testing the creation and saving of a blank UIManagedDocument. The completion handler block is invoked when the operation completes and we test for a True value of the “success” block parameter. If we run the test from Xcode it will pass. It will always pass, and never fail. That’s because the test exits before the completion handler is invoked. We can check this is so by setting a breakpoint on the following line:

STAssertTrue(success, @"Should have been success!");

Run the test. The breakpoint is never reached and so we can never test the value.

So how can we ensure that the test method waits until the asynchronous method invokes the completion block? There have been a few solutions proposed on places like Stack Exchange, and after playing with a few, I use the following one.

In this solution, we need to declare a Boolean flag that indicates that we’re waiting for the operation to complete, then call the asynchronous method, and then block the run loop until the test completes when the flag is set to NO.

Here’s how we do it. We need to declare our Boolean flag before calling the asynchronous method:

__block BOOL waitingForBlock = YES;

Then we must set it to NO inside the method’s completion block:

waitingForBlock = NO;

And loop while the condition is true:

while(waitingForBlock) {
    [[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode
                             beforeDate:[NSDate dateWithTimeIntervalSinceNow:0.1]];
}

Here’s the full code:

- (void)testSaveAndCreateDocument {
    NSURL *url = ...; // URL to file
    UIManagedDocument *document = [[UIManagedDocument alloc] initWithFileURL:url];

    // Set the flag to YES
    __block BOOL waitingForBlock = YES;

    // Call the asynchronous method with completion block
    [document saveToURL:document.fileURL
        forSaveOperation:UIDocumentSaveForCreating completionHandler:^(BOOL success) {
            // Set the flag to NO to break the loop
            waitingForBlock = NO;
            // Assert the truth
            STAssertTrue(success, @"Should have been success!");
        }];

    // Run the loop
    while(waitingForBlock) {
        [[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode
                                 beforeDate:[NSDate dateWithTimeIntervalSinceNow:0.1]];
    }
}

Run the test again, and this time the breakpoint is reached.

This seems somewhat cumbersome, especially if we need to do this in many Unit Tests. So we can convert this into a set of Macros and declare them in a supporting header file:

// Set the flag for a block completion handler
#define StartBlock() __block BOOL waitingForBlock = YES

// Set the flag to stop the loop
#define EndBlock() waitingForBlock = NO

// Wait and loop until flag is set
#define WaitUntilBlockCompletes() WaitWhile(waitingForBlock)

// Macro - Wait for condition to be NO/false in blocks and asynchronous calls
#define WaitWhile(condition) 
do { 
    while(condition) { 
        [[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode beforeDate:[NSDate dateWithTimeIntervalSinceNow:0.1]]; 
    } 
} while(0)

Now we can simplify our test:

- (void)testSaveAndCreateDocument {
    NSURL *url = ...; // URL to file
    UIManagedDocument *document = [[UIManagedDocument alloc] initWithFileURL:url];

    // Set the flag
    StartBlock();

    // Call the asynchronous method with completion block
    [document saveToURL:document.fileURL
        forSaveOperation:UIDocumentSaveForCreating completionHandler:^(BOOL success) {
            // Set the flag to NO to break the loop
            EndBlock();
            // Assert the truth
            STAssertTrue(success, @"Should have been success!");
        }];

    // Run the Wait loop
    WaitUntilBlockCompletes();
}

Using the Macros makes things much clearer. We’re simply initialising a flag to YES with the StartBlock() pseudo-function, setting it to NO with EndBlock() and waiting for the flag to be set to NO in the WaitUntilBlockCompletes() call.

It’s not the most elegant solution, but it suffices for my needs.

I’ve uploaded the Macros to GitHub as a Gist:

https://gist.github.com/Phillipus/6537635

A caveat – sometimes this solution does not work using Xcode 5 with iOS 7, especially when dealing with a UIManagedDocument. In some cases there seems to be a race condition or threading issue going on. (See the workaround in the comments for UIManagedDocument.) Also, with the iOS 7 simulator, I’m seeing more and more cases where this wait loop is not working, and some unit tests are not being run. It’s as if some tests are not waiting for the loop to break and failing to run. I’ve tried increasing the value of dateWithTimeIntervalSinceNow in this line of the macro:

[[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode beforeDate:[NSDate distantFuture]];

But it seems as if the main UI thread seems to get stuck in some tests, and can only be unstuck by clicking on the simulator, or waiting a few seconds. Something has changed in Xcode 5 and/or iOS 7, but I don’t know what it is. If you have problems with this, experiment with the date value in this line. It may just be an issue with the simulator. Try testing on the device as well.

I still have my reservations about this technique, and I’m still looking for the perfect solution for asynchronous unit testing in Xcode. You would think that Apple might have provided a solution in XCTest, perhaps similar to the implementation in GHUnit.

Update 14 July 2014 – Good news! Apple has introduced a great new framework to support asynchronous unit testing in Xcode 6 (still in beta at the moment). I’ve written a new post on how to implement this here.

GitHub Gists

PhillipusAugust 20, 2013May 19, 2021

Yes, I know I’m late to the party. I’ve been using CodeBox for all of my code snippets, but today I created my first GitHub Gist. Easy to maintain and share. And a bonus – it’s damn easy to embed a Gist in a WordPress blog. Just add the URL and you’re done. Interesting thing, though – if you revise and change the Gist, the blog post could get out of sync with the code. Can you link to a specific revision of the Gist?

Here it is

(Edit – I removed the embedded gist plug-in)