Lessons from EAF

4. Abstract data by type

When we built the ElseApps Framework we learnt a lot of lessons. This series of short articles discusses some of the key learnings from our framework development. I’m sure most of them will be lessons other developers have already learnt, but many of them weren’t in our vocabulary when we started into the project.

fw_ptformWe’re no strangers to writing software for multiple organsations in the same business space. Before we wrote the ElseApps Framework we’d developed several discrete web applications for contestable funding organisations. When developing for an individual client, data structures can directly reflect the client’s business process, but this isn’t the case when you’re writing a framework for re-use across multiple organisations – or even dealing with a single organisation which has complex requirements.

Consider the client with twelve different data flows, each of which requires separate sets of forms. Sure, there’s no problem creating twelve sets of static HTML forms, but how do you normalise the data?

From experience, some fields will shared across multiple data streams, others will only be used in a single stream. Attempting to shoehorn all the resulting data into a single table will result in an unwieldy data structure and a lot of conditional rules within the internal data management processes. Multiply this scenario across half a dozen clients, and you no longer have a customisable framework, but six bespoke applications.

Java evangelists are fond of the phrase “Write once, run anywhere”. We prefer “Write once, go home”.

fw_formeditor02In designing EAF we elected to abstract data by type. Forms of any type used within the framework are created using an editor built into the framework itself and stored in a series of database tables that describe the form, its sections and layout, its fields and their datatypes.

Currently our forms support twenty different field types including multi-selects, typeahead lookups and file uploads. Each field type has its own set of options, and all of them support a set of common responsive layout directives. The HTML forms and their accompanying Javascript helpers are generated dynamically from the parameters and options selected in the editor.

This approach delivers consistently good-looking and user-friendly forms for any purpose. Not only does it allow us to generate an infinite number of forms with ease, it also requires no additional coding to populate and save each form: there’s a set of standard classes in EAF’s core that takes care of this, including client-side validation of mandatory fields and server-side verification of the submitted data.

But that’s not all – EAF also stores data by type.

fw_datafieldsEach form type – projects, submissions, users, reports, whatever – has a stub table in the database that contains primary and foreign keys. And then there’s a single set of database fields to which all other data is mapped by datatype. Our core code maps each form field to the appropriate data field. This makes data storage extremely efficient on space.

We take this paradigm one step further with text fields. We dynamically assign them to one of three fields depending entirely on the length of the text: less than 32 characters, less than 512 characters, anything larger. Again, this ensures efficient data storage, and the core framework robustly manages the work of moving text data between these three tables as the length of that data changes.

The result of this approach is a high level of flexibility within the core framework before we ever have to think about customisation.

  • Multiple forms on one page? No problem.
  • Dynamic form sections? No problem.
  • Forms in modal dialogs? Nae problemo.

As discussed in a previous lesson we’ve designed the framework to make customisation simple, but the cost/benefit equation tilts even further in EAF’s favour when no customisation is required.

Our next lesson from EAF? 5. Generate consistent output.

Lessons from EAFF: 1 2 3 4 5 6 7 8