The evolution of web service protocols pt 1

At San Digital, some of us have been building things for people since the dawn of the web. Our historical perspective helps inform us about technological culture and trends today, almost compensating for the creaking knees.

The primordial era

The web before 1998 was a simple but chaotic place, inhabited by a few creatures of myth and legend. Standards were minimal and web server implementations largely proprietary. Browsers unable to do much more than synchronously post a form. Servers, usually limited to the Common Gateway Interface - a simple yet effective convention for executing malicious shell scripts on the beige box under the desk of those enthused by neologisms like "Information Superhighway".

This new protocol - Http, had promise for more than serving under construction GIFs and pictures of a filter coffee machine. It could be used to interact with services located under desks anywhere in the world; revolutionary at the time. People began to experiment with ways of improving machine to machine communication concurrently with rapid developments in browser and server technology.

Would you like some Xml with your Xml?

Machine to Machine communications in this era were dominated by various forms of RPC, terrifyingly complex object invocation protocol's like CORBA, and Microsoft's CORBA, DCOM. There were no real open standards at this point, and the object protocols were barely usable on a corporate LAN, never mind when communicating to an intermittently available whale fat powered server somewhere north of Tromsø. Http had been designed to be simple and robust enough to be able to cope with the awful reliability and latency of the early internet, you could mostly GET stuff you were interested in and you could mostly POST stuff you wanted to tell people about; the question now was what?

The H in both Html and Http is Hypertext. Html, an at the time simple markup language perfectly suited for the purpose of creating brutalist "home pages" for computer scientists and other inhabitants of the early web, as yet untainted by the ambitions of User Experience architects and Javascript developers. Html was derived from SGML, an ambitious standard to unify document markup languages, and had an ambitious sibling - Xml.

Xml had a difficult childhood, it was conceived by a standards body who wanted it to be both a document markup language and also an interoperation standard for data, releasing 1.0 of the standard in 1998. This confused upbringing was however a perfect match for the web - the same representation could potentially be used to render Dave Weiner's homepage in a browser and for a computer program to find out what the weather was like in Tromsø.

Microsoft - at the time much and sometimes deservingly maligned company commonly known to web dwellers at the time by hilarious nicknames like 'M$' had begun to take an interest in the potential of Http for services and along with Dave Wiener conceived of a new standard that ..utilised Http called Xml/RPC. With Xml/RPC, we now had the beginnings of a workable and interoperable way of POSTing things that were not Html forms over the internet. Such was the enthusiasm for POSTing things that many forgot about GETting things entirely, an issue that will rear its head later in our story.

An Xml/RPC invocation

This request:

<methodCall>
  <methodName>cabin.getWeather</methodName>
  <params>
  </params>
</methodCall>

Should respond with:

<methodResponse>
  <params>
    <param>
        <value><string>☃︎</string></value>
    </param>
  </params>
</methodResponse>

Or more probably:

<methodResponse>
  <fault>
    <value>
      <struct>
        <member>
          <name>faultCode</name>
          <value><int>4</int></value>
        </member>
      </struct>
    </value>
  </fault>
</methodResponse>

Xml/RPC was short lived outside of Dave's house, but at this point Don Box enters the story - a long time community COM expert and the coiner of one of the most jarring idioms in software history - "COM is Love". A phrase that most of those whose COM subject matter expertise was insufficient to pay for their children to go to collage would probably disagree with. Microsoft, and Don began to work on improvements to the Xml/RPC standard that would eventually result in SOAP, allegedly despite much internal competition from the COM team.

Jumping into the FHIR - type systems and objects

We have been doing a deep-dive on FHIR implementations and tooling following our initial FHIR investigation. A critical area of investigation for any system, particularly a large distributed system with many clients and peers that need longevity and guided evolution is its type system. Use of a strict type system can have many benefits - interoperability, governance, data quality, security, performance, developer experience, reduced component complexity and the ability to evolve services with confidence

Integrating with Events

The San Digital team has worked with numerous organisations in both the public and private sectors to transform their applications architecture into a flexible and business-focused model. Working with events at scale is key to maintaining individual teams' agility.

The process of building a mobile app

The team at San Digital has extensive experience developing apps for mobile devices, smartwatches, and smart TVs; using native and hybrid technologies (and everything in-between!) including using Rust for complex comms.

Low friction development environments

While setting up a sample project from an unnamed large vendor the other week I was disappointed by having to read large amounts of documentation and run various bits of script to install dependencies and set up infrastructure. We live in a world that has tools old (Make) and new (Docker) that can be combined to make onboarding engineers low or zero friction.

Cloud-native FHIR platforms

Continuing our series of posts on web protocols, we have been investigating more specialist protocols, in this case, "FHIR". We have produced a document based on our research, investigations and experience.

Team Structures

Multiple team structures can work to deliver software projects. There is no real one size fits all, however, there are common components that can be seen across different structures. At San Digital we believe that Engineer-led teams deliver great results for short duration high-impact projects.

Rules of the Road

This is called rules of the road but they aren't rules they're more guidelines, so they're rules until there is a good reason to ignore them.

Estimating and delivering defined outcomes

Recently there has been a shift away from time and materials projects towards defined outcomes, driven by various legislative changes, specifically IR35, but also cost control in the procurement function of larger organisations.

The San Digital Stack

San Digital has been designed as a remote first business from inception, on the assumption that it's easier to add offices later if they are necessary in an agile way. To work in collaborative way completely remotely takes a carefully thought out set of tools. Some of the ones that we use are really standard and some are a little more interesting.

Test driven design, or planning driven development

Design processes in most business software development resemble peer review or crowd-sourcing. A putative design is presented to peers, who will do their best to find problems while the originator of the design defends it against these challenges. Ideally, where they are demonstrated incorrect or incomplete the process will iterate and an updated design produced and defended.

A human view of computer vision at scale

Computers analysing and acting on what they see is not science fiction or even a new concept, it has been a reality of humankind's drive towards hyper-efficiency since around the time I was born.

Building scalable frontends

Scaling frontends is hard, actually scaling all codebases is hard, frontends just happen to be particularly visible and have a tighter feedback loop and a higher rate of change. As with all codebases, it is in principle possible to scale development through standards and integration processes, but these are a poor substitute for communication. Once development moves beyond the scope of a single team, either progress slows to take into account of different processes or implementations drift away from each other over time. Teams need to find a way to operate independently towards a goal.

Cross platform native mobile development with Rust

San Digital have extensive experience of mobile development and the use of Android as an embedded operating system. We treated android as a deployment target target for Rust firmware as well as writing our intricate real time communications component for both iOS and Android. This approach has advantages, you can maintain a single code base for a complicated communications layer, while also taking advantage of the full native capabilities of each platform

The evolution of web service protocols pt 2

At San Digital, some of us have been building things for people since the dawn of the web. Our historical perspective helps inform us about technological culture and trends today, almost compensating for the creaking knees.

The evolution of web service protocols pt 1

At San Digital, some of us have been building things for people since the dawn of the web. Our historical perspective helps inform us about technological culture and trends today, almost compensating for the creaking knees.