Flutter – Scrolling Content

I didn’t see this post coming.

I had planned to talk about adapting the app for display on the iPad and realised I needed to start by creating a recipe details screen for the iPhone.

Recipe details screen design
Recipe details screen design

Unfortunately, the content didn’t scroll which caused a display issue when the number instructions or ingredients increased.

Overflow error when too many ingredients
Overflow error when too many ingredients

Ta Da

After a bit of research (see links below) I found the way to implement a natural iOS experience where the content scrolls under the tab and navigation bars.

Recipe details before scrolling
Recipe details before scrolling
Recipe details after scrolling to the bottom
Recipe details after scrolling to the bottom

BackBurner

We haven’t spent any time improving the style of the recipe details screen, it’s pretty bland, not something I’d want to release. We can tackle this in a future post, once we have a few more screens up and running.

Large navigation bar title that shrinks when the content is scrolled. This can be achieved when you use the CupertinoSliverNavigationBar as the top widget in the sliver list, it requires largeTitle or it errors, so you cannot use it as the normal navigation bar. The plan is to add it as extra functionality to the new TabScrollableContentScaffold widget.

Large title in the navigation bar.
Large title in the navigation bar.
Nav bar title has shunk and moved up after scrolling
Nav bar title has shunk and moved up after scrolling

XP

We needed to slow down and make sure that we could provide a native experience when scrolling content in a tab.

Which includes content appearing to scroll under the tab and navigation bar and for it to bounce and snap when it reaches the beginning or end. This was abstracted into a widget called TabScrollableContentScaffold so we can easily reproduce the experience on other tabs we build.

@override
Widget build(BuildContext context) {
  
  var appState = Provider.of<DigestableMeState>(context);

  return TabScrollableContentScaffold(
    navigationBar: navBar(),
    children: [
      recipeTitleAndAuthor(appState),
      recipeImage(),
      for (Widget item in ingredients(appState))
        item,
      for (Widget item in instructions(appState))
        item,
      shrug()
    ],
  );
}

Mapping hurts, is costly, never under estimate it, get good at map -> reduce in the languages you use. Had to add some more code and tests to fix the mapping from ‘serpApiResult’ results to the ‘Recipe’ view model.

You may have noticed that the initial screen design changed, instructions and ingredients are not side by side, this sort of small change is the normal during most implementations, just remember to demo and agree them.

Links

Sound & Vision

The shocking Miss Emerald, by Caro Emerald.

One more thing

You go to your TV when you want to turn your brain off. You go to your computer when you want to turn your brain on. Those are not the same.

Steve Jobs_

Flutter – Themes

Lets theme the the application, make it look good, do it in a maintainable way that can be easily rebranded.

I started by adding a ‘light’ theme in its own file:

Cupertino light theme files
Cupertino light theme files
import 'package:flutter/cupertino.dart';

const CupertinoThemeData cupertinoLight = CupertinoThemeData(
  brightness: Brightness.light,
  primaryColor: CupertinoColors.activeBlue
);

And using it in the main.dart file:

return CupertinoApp(
  ...  
  theme: cupertinoLight,
  ...
);

We can then remove the colour settings that were added to get the initial green background for the tab.

Digests tab - Cupertino Light Theme
Digests tab – Cupertino Light Theme
Recipe Search - Cupertino Light Theme
Recipe Search – Cupertino Light Theme

Ok, what about a dark theme ?

import 'package:flutter/cupertino.dart';

const CupertinoThemeData cupertinoLight = CupertinoThemeData(
  brightness: Brightness.dark,
  primaryColor: CupertinoColors.activeGreen
);
Digests tab - Cupertino Dark Theme, missing text
Digests tab – Cupertino Dark Theme, missing text

Black text on a black background a little disappointing.

To fix this I created the text styles needed and added a switch(bool) to the application state file ‘DigestableMeState’ to control the switching between the light and dark modes.

Digests tab - Cupertino Dark Theme
Digests tab – Cupertino Dark Theme
Recipe search - Cupertino Dark Theme
Recipe search – Cupertino Dark Theme

Ok, now we look like some of Apple’s iOS applications, but what if we want to rebrand our application for EE ?

What is a white label?

In mobile applications you don’t usual need to include the logo in the application screens as its identity is the app you install, so we can just concentrate of a swatch of colours and the typography.

EE Swatch & Typography
EE Swatch & Typography

A few more files and constants later…

EE Colours
EE Colours
Provider(
	create: (context) => DigestableMeState(
    	brightness: Brightness.dark,
        whiteLabel: WhiteLabel.ee
    ),
),

and we launch as EE in evening dark mode.

Digests tab - EE Dark Theme
Digests tab – EE Dark Theme
Recipe search - EE Dark Theme
Recipe search – EE Dark Theme

It is really about being flexible to change styles at the application level without having too may specific styles in the widgets themselves.

BackBurner

  • Automatically toggling between light and dark mode based on the current device setting.
  • Handling themes across platform to include Android or the web.
  • Custom widgets themes for custom widgets, not even sure if they are necessary at this point so something to come back to. A custom widget theme would allow you to target the custom widgets with styles i.e. we could target the all ListItem widgets to add border, padding or other styles as the application level.

XP

Getting the dark theme to work required more effort than just having a top level theme.

I had to remove the Card from the ListItem widget and go back to simple Rows, Columns, which is really what I should have used in the first place.

I replaced text styles that I had specified in the widgets

style: const TextStyle(
	color: Colors.black87,
    fontSize: 14,
),

With text styles that I created to support the Apple Typography guidelines:

Apple default text styles
Apple default text styles

Injection:

Provider(
	create: (context) => DigestableMeState(
    	brightness: Brightness.light,
    ),
)

Construction:

DigestableMeState({Brightness? brightness}) {
    if (brightness != null) {
      this.brightness = brightness;
    }
    _cupertinoTypography = CupertinoTypography(this.brightness);
...

Definition:

CupertinoTypography(this.brightness);

TextStyle bodyLight = const TextStyle(
  color: CupertinoColors.black,
  fontSize: 17,
);

TextStyle bodyDark = const TextStyle(
  color: CupertinoColors.white,
  fontSize: 17,
);
TextStyle get body {
  return brightness == Brightness.dark ? bodyDark : bodyLight;
}
...

Usage:

var appState = Provider.of<DigestableMeState>(context);
Text(item.title, style: appState.cupertinoTypography.title3)

Links

Sound & Vision

It’s only love with a little be of rain…

Carnage by Nick Cave & Warren Ellis.

One more thing

“Older people sit down and ask, ‘What is it?’ but the boy asks, ‘What can I do with it?’”

Steve Jobs_ on showing a 9-year-old a Mac

Great quote, reminds me to play with new tech, make mistakes, learn, rather than try to define or control it.

Flutter – Adding an Item

Each year we try to make an improvement to our dinner on Christmas day, this year the focus is on better roasties.

Let’s look at how we can find and add a new roast potato recipe to our digest.

The acceptance for this requirement is to be able to:

  • Search for a new recipe
  • Save it to our digest:
    • Title
    • Author
    • Thumbnail image

To get started I added a top navigation bar with an action that allows us to add a recipe.

@override
Widget build(BuildContext context) {
  return CupertinoPageScaffold(
    navigationBar: const CupertinoNavigationBar(
      heroTag: "RecipeTabHeroTag",
      transitionBetweenRoutes: false,
      middle: Text(
        "Recipies",
      ),
      trailing: Icon(CupertinoIcons.add)
    ),
    child: Container(
      color: CupertinoColors.activeGreen,
      child: recipiesList(),
    ),
  );
}
Shows the Cupertino scafold and App bar used to display our recipes
Shows the Cupertino scafold and App bar used to display our recipes

I renamed the tab back to ‘Digests’ now we have a title of ‘Recipes’ in the top navigation bar.

Now for another quick decision we are going to slide up the screen when the add button is clicked and show a search bar to allow us to find new recipes.

Shows the recipe search bar
Shows the recipe search bar

With that out the way we just need to harness Google search to find some new recipes.

I turned to SerpAPI for this and grabbed some data from a recipe search for roast potatoes with balsamic vinegar.

https://serpapi.com/search.json?q=Roast+potatoes+with+balsamic&location=United+Kingdom&hl=en&gl=uk&google_domain=google.co.uk&api_key=secret_api_key

{
    "search_metadata": {
      "id": "61c3369cc99903747c1e643b",
      "status": "Success",
      "json_endpoint": "https://serpapi.com/searches/bdb5db8a7f4c5593/61c3369cc99903747c1e643b.json",
      "created_at": "2021-12-22 14:30:52 UTC",
      "processed_at": "2021-12-22 14:30:52 UTC",
      "google_url": "https://www.google.co.uk/search?q=Roast+potatoes+with+balsamic&oq=Roast+potatoes+with+balsamic&uule=w+CAIQICIOVW5pdGVkIEtpbmdkb20&hl=en&gl=uk&sourceid=chrome&ie=UTF-8",
      "raw_html_file": "https://serpapi.com/searches/bdb5db8a7f4c5593/61c3369cc99903747c1e643b.html",
      "total_time_taken": 1.99
    },
    "search_parameters": {
      "engine": "google",
      "q": "Roast potatoes with balsamic",
      "location_requested": "United Kingdom",
      "location_used": "United Kingdom",
      "google_domain": "google.co.uk",
      "hl": "en",
      "gl": "uk",
      "device": "desktop"
    },
    "search_information": {
      "organic_results_state": "Results for exact spelling",
      "query_displayed": "Roast potatoes with balsamic",
      "total_results": 12800000,
      "time_taken_displayed": 0.49
    },
    "recipes_results": [
      {
        "title": "Balsamic potatoes",
        "link": "https://www.jamieoliver.com/recipes/potato-recipes/balsamic-potatoes/",
        "source": "Jamie Oliver",
        "total_time": "2 hrs 20 mins",
        "ingredients":
        [
          "Balsamic vinegar",
          "maris piper potatoes",
          "red onions",
          "rocket",
          "olive oil"
        ],
        "thumbnail": "https://serpapi.com/searches/61c3369cc99903747c1e643b/images/bd928f9ef521c02bbdb2df96157011d6a02d619d06f8a6e0e62bb157f48e10e5.jpeg"
      },
      {
        "title": "Balsamic roast potatoes",
        "link": "https://www.taste.com.au/recipes/balsamic-roast-potatoes/1bcb6bd4-2efa-4f01-bc98-f2ce32eaa906",
        "source": "Taste",
        "rating": 4,
        "reviews": 2,
        "total_time": "1 hr 5 mins",
        "ingredients":
        [
          "Balsamic vinegar",
          "kipfler potatoes",
          "olive oil",
          "garlic"
        ],
        "thumbnail": "https://serpapi.com/searches/61c3369cc99903747c1e643b/images/bd928f9ef521c02bbdb2df96157011d625322c9b727d95f76706f0471bbaba31.jpeg"
      },
      {
        "title": "Balsamic roast potatoes",
        "link": "https://www.delicious.com.au/recipes/balsamic-roast-potatoes/1e2f30ad-d9b4-41bf-aa2e-16f1e2accfb7",
        "source": "Delicious",
        "rating": 5,
        "reviews": 1,
        "total_time": "1 hr 5 mins",
        "ingredients":
        [
          "Balsamic vinegar",
          "kipfler potatoes",
          "olive oil",
          "garlic"
        ],
        "thumbnail": "https://serpapi.com/searches/61c3369cc99903747c1e643b/images/bd928f9ef521c02bbdb2df96157011d6ebca3a2868589cb9feab32215a0ba5e8.jpeg"
      }
    ]
}

Added a list to return the results from the search.

The results from a recipe search for roast potatoes with balsamic vinegar
The results from a recipe search for roast potatoes with balsamic vinegar

When a result is selected (onTap) we add it to our recipe digest.

Ta Da

Shows the recipe digest after the new recipe is added.
Shows the recipe digest after the new recipe is added.

Back Burner

Saving the digest.

For now we save the added recipe in memory, later we can persist it locally and remotely to our data stores.

iOS Only – Click tab to reset

Adding functionality to navigate back to the top level/state when tapping on the current tab item, this is a good iOS experience, a design that people expect.

Animated Search Bar

I decided to avoid implementing a more advance search bar similar to the one in Apple’s “App Store” application that has animation on focus.

An example of a search bar with animation on focus used by Apple's 'App Store" app.
An example of a search bar with animation on focus used by Apple’s ‘App Store” app.

We can look at this and attempt to improve the search bar later on once the basics are in shape.

CupertinoSearchTextField

I’m getting some library level errors from Flutter asking me to report a framework bug when hitting enter on the keyboard in the search field using a iOS emulator.

For now I’ve abstracted it into a widget and intend to replace it with a custom version with more features.

I’ve also noticed that you need to enter twice on the keyboard in the emulator before the ‘OnTap’ event fires.

XP

In memory data

We need to store and maintain a list of recipes to allow us to add new recipes.

At this point are using a memory repository class to manage this and injecting it as a ChangeNotifierProvider.

ChangeNotifierProvider<MemoryRepository>(
	create: (_) => MemoryRepository(),
    lazy: false,
),

It sounds complicated but it’s just a list of recipes in memory and some methods to add and retrieve them.

List<Recipe> findAllRecipes();
Recipe addRecipe(Recipe recipe);

See the Flutter documentation for more information on state management.

When we move to a data store e.g. a DB via an API we can improve this code.

List Item Reuse

As part of displaying the search results we need to display a list again, so I abstracted the initial digest list item so we could reuse it and ensure our list items have the same look and feel across the app.

Test results for the list item widget
Test results for the list item widget
Digest list item
Digest list item
Widget recipesList() {
  final repository = Provider.of<MemoryRepository>(context);
  var recipes = repository.findAllRecipes();

  return ListView.builder(
    itemCount: recipes.length,
    itemBuilder: (context, index) {
      var listItemInfo = ListItemInfo(
        id: recipes[index].link,
        author: recipes[index].source,
        title: recipes[index].title,
        titleImage: Image.network(recipes[index].thumbnail),
        showAddIcon: false,
      );

      return ListItem(listItemInfo);
    },
  );
}
Recipe search result list item
Recipe search result list item
Widget recipeSearchResultsList() {
  var searchResultRecipes = _searchResults.data.length > 0
    ? Recipe.fromSerpApiResultJson(_searchResults.data)
    : <Recipe>[];

  final repository = Provider.of<MemoryRepository>(context);

  return ListView.builder(
    itemCount: searchResultRecipes.length,
    itemBuilder: (context, index) {
      var listItemInfo = ListItemInfo(
        id: searchResultRecipes[index].link,
        author: searchResultRecipes[index].source,
        title: searchResultRecipes[index].title,
        titleImage: Image.network(searchResultRecipes[index].thumbnail),
        showAddIcon: true,
        onTap: () => {
          repository.addRecipe(searchResultRecipes[index]),
          Navigator.pop(context)
        },
      );

      return ListItem(listItemInfo);
    },
  );
}

Scope Creep (Beware – YUTNI Your unlikely to need it)

For me this was the most tempting time to try and over engineer the project as it is still missing a number of key things.

I wanted to do more work on:

  • data access, including batched api requests
  • design much better widgets with nice animation
  • add a theme and a flavour
  • logging
  • Automated acceptance testing.
  • Improve the design for Android
  • Look at how it would work as website
  • bring in some state management libraries
  • etc..

But experience is telling me to resist the urge to get side tracked, even some initial research on google, has slowed this post down considerably.

In the end I went back to the demo mentality of get it working, balanced with advice from Uncle Bob, that once it’s working that’s only 50%, you now need to get it right.

That equated to some more time extracting methods and tidying up.

As long as we give each class, method a single responsibility in its own file, we can easily improve them independently later on, as we move from concept to production ready.

Often the scope creep comes in when I get bored with the discipline of the task at hand as disciplines feel slow.

Links

A big thanks to M4trix Dev as his article on the iOS Tab Bar was invaluable and allowed me to quickly fix the problems I had with my first implementation.

People

“The thing is, it’s very easy to be different, but very difficult to be better.”

Jony Ive_

Picture of Jony Ive

Sound & Vision

The Bonny - Gerry Cinnamon

Summary of BBC drama Around the world in 80 days

Title image from BBC drama around the world in 80 days.

One more thing

“I have looked in the mirror every morning and asked myself: “If today were the last day of my life, would I want to do what I am about to do today?” And whenever the answer has been “No” for too many days in a row, I know I need to change something.”

Steve Jobs_

Flutter – Lists

With xmas round the corner, our sponsor has asked that we add a new digest to share recipes.

Santa’s helper will need to get up and have a pre-breakfast boiled egg, so lets just cut to to the chase and start with some actual data, some json:

{
	"title": "How to boil an egg, the Heston Blumenthal way",
	"titleImage": "https://i.guim.co.uk/img/static/sys-images/Guardian/Pix/pictures/2014/11/5/1415205733799/4bfbd71a-6cd0-4494-833f-eaaed20a15b3-1020x612.jpeg?width=620&quality=45&auto=format&fit=max&dpr=2&s=ca3a95d7e761d267eff1b79b58cc4849",
	"author": "Heston Blumenthal",
	"origin": "https://www.theguardian.com/lifeandstyle/2014/nov/11/how-to-boil-an-egg-the-heston-blumenthal-way",
	"ingredients": [
        {
        	"quantity": "1",
			"quantityType": "unit",
            "name": "egg",
			"type": "Dairy"
        }
	],
	"steps": [
        {
        	"sequenceNumber": "1",
            "instruction": "Take a small saucepan with a glass lid and carefully place a single egg (or two, or three) inside it. Burford brown eggs have a nice orange yolk. Fill the pan so the water only just covers the eggs – not even a millimetre more. If you had a centimetre of water covering the egg then you could still get the same result, but you’d have to play with the timing."
        },
        {
        	"sequenceNumber": "2",
            "instruction": "Put the pan on maximum heat with the lid on and bring to the boil."
        },
		{
        	"sequenceNumber": "3",
            "instruction": "As soon as the water starts to bubble, remove from the heat. As you take the pan off, set a timer for six minutes; keep the lid on. Make sure you time it exactly, and you’ll end up with the perfect egg."
        }
	]
}

Some acceptance criteria:

  • Displays a list of recipes
  • Each recipe has a title, author and a picture.

And a design for our list item:

Recipe list item design
Recipe list item design

Because it’s so close to Christmas we couldn’t get our API approved, so we created a simple api client class that mocks the data for the recipe list.

Now we have the data we can display the data items in a list using the ListView widget.

Ta Da

Recipe List
Recipe List

You maybe thinking why didn’t we build the digest list first ?

Because we only have one interest, recipes, we can add the digest list when it is needed.

The focus is on building content that matters and not framework or things we might need.

So on that note, for now, lets just change the tab description:

Recipe Tab Descriptionrecipies_tab.dart

BackBurner

Use of a API on a server, instead of mocked data.

Better application of styles, including inclusion of Apples design guideline for the TabBar, that state that it should be translucent when items are under it and opaque when at the end of the list.

XP

Use of dio a powerful Http client for Dart, which supports Interceptors, used to inject canned data.

The reason for choosing and using a package now is that it saves us having to create requests and response objects, we will hook it up to a real Api soon.

People

“The speed of the project is the speed at which ideas move between minds”

“or its inverse which is stronger, anything that slows the movement of ideas between minds slows the project down”

 Alistar Cockburn_

One more thing

“You can’t connect the dots looking forward; you can only connect them looking backward. So you have to trust that the dots will somehow connect in your future.”

Steve Jobs_

Restful Api’s

Here’s my way, my influences, but feel free to follow any highway…

Naming

Resources are objects, things, so use a noun to specify a resource e.g clients and then use HTTP methods to define actions like get, modify, delete.

If you use verbs like send, get then you are falling into the Remote Procedure Call trap. RPC style endpoints are notorious for specifying the function in the URL e.g. /getClientNameById

Resource names should be plural as this is the most widely adopted approach:

/clients not /client /getClients /user/client

Unless there can only ever be one (0-1, exists or not) e.g. users/1/avatar.

HTTP Methods

Often call HTTP verbs are POST, GET, PUT, PATCH, DELETE and OPTIONS.

Considering the resource (Entity) life cycle can help you determine which HTTP methods to allow, in particular where it is stored, db, file and if it can be archived.

Patch & Put

  • Favor Patch for updating resources, use put when resources are replaced not updated e.g. binary documents.
  • Do not use Put to create new items.

    Be aware that PUT does a complete overwrite of the data,
    it is a request to replace a resource, PATCH is a request to
    update part or all of it i.e. a new version.

The difference between the PUT and PATCH requests is reflected in the way the server processes the enclosed entity to modify the resource identified by the Request-URI.

In a PUT request, the enclosed entity is considered to be a modified version of the resource stored on the origin server, and the client is requesting that the stored version be replaced. With PATCH, however, the enclosed entity contains a set of instructions describing how a resource currently residing on the origin server should be modified to produce a new version.

PATCH is less likely to have side affects when a resource is updated frequently because you only update the things that change, whereas PUT will update fields that have not changed with the values retrieved and if another request happens after the retrieval you will reset them. You can/should guard against this at the repository level by checking a resource version and invaliding the update.

Do not use ‘Put’ to create new items, use ‘Post’.

If barn 11 does not exist then PUT /barn/11 should return a 404 and message saying I couldn’t modify it because it doesn’t exist.

Options Verb

The OPTIONS method represents a request for information about the communication options available on the request/response chain identified by the Request-URI. This method allows the client to determine the options and/or requirements associated with a resource, or the capabilities of a server, without implying a resource action or initiating a resource retrieval.

A pre-flight options request is triggered by some CORS requests, see the section on CORS for more details.

Response Codes

Always return a response code for each request.

HTTP/1.1 200 OK
Date: Sun, 18 Oct 2012 10:36:20 GMT
Server: Apache/2.2.14 (Win32)
Content-Length: 230
Connection: Closed
Content-Type: text/html; charset=iso-8859-1
Response Codes

If I did GET or an update, 200 is okay.

However, if I do POST or a PUT and something is created, use 201. Tell me explicitly that the new object was created.

If I do PUT or a PATCH and nothing’s modified, return 304 Not Modified.

If I send the wrong data or use the wrong format request, return 400 Bad Request.

If I haven’t logged in or I sent an invalid auth token, return 401 Not Authorised.

If I try to do something I’m not allowed to do, 403 Forbidden.

404 if the object never existed, or it’s not there. Technically, you want to use the status code for Gone namely, 410 if it once existed, but 404 is traditionally Not Found. DO NOT use this when a GET returns no rows/result, it’s still a 200.

405 represents Method Not Allowed. This goes beyond Forbidden 403 and says, “Hey, you can’t do this for example, delete an object. If you try a different method, that would work.”

415 corresponds to Unsupported Media Type, for example if I request XML but you only support JSON.

Formatting Content

I follow the conventions in the { json;api }  specification.

// Articles with fields title, body and author.
GET /articles?include=author&fields[articles]=title,body,author

HTTP/1.1 200 OK
Content-Type: application/vnd.api+json

{
  "data": [{
    "type": "articles",
    "id": "1",
    "attributes": {
      "title": "JSON:API paints my bikeshed!",
      "body": "The shortest article. Ever."
    },
    "relationships": {
      "author": {
        "data": {"id": "42", "type": "people"}
      }
    }
  }]
}

The spec has matured overtime and specifies some of our important practices showing us how to format the negotiation between the client and server:

  • Relationships
  • Hypermedia
  • Pagination
  • Filtering
  • Sparse fields
  • Errors

Relationships (Related Resources)

Multiple related resources can be requested in a comma-separated list:

GET /articles/1?include=author,comments.author HTTP/1.1
Accept: application/vnd.api+json

In order to request resources related to other resources, a dot-separated path for each relationship name can be specified:

GET /articles/1?include=comments.author HTTP/1.1
Accept: application/vnd.api+json

To update a related resource include the resource as a relation in the PATCH e.g. request will update the author relationship of an article:

PATCH /articles/1 HTTP/1.1
Content-Type: application/vnd.api+json
Accept: application/vnd.api+json

{
  "data": {
    "type": "articles",
    "id": "1",
    "relationships": {
      "author": {
        "data": { "type": "people", "id": "1" }
      }
    }
  }
}

Hypermedia

Hypermedia As The Engine Of Application State.

Richardson Maturity Model
Richardson Maturity Model

The term “hypermedia” was coined back in 1965 by Ted Nelson, and over the years has dominated the technology industry. Hypermedia, in its most basic sense is an extension of hypertext – something you may recognise from HTML.

Hypertext is essentially text that is written in a structured format and contains relationships to other objects via links.

Hypermedia is just an extension of the term hypertext, hypermedia includes images, video, audio, text, and links.

In a REST API, this means that your API is able to function similarly to a web page, providing the user with guidance on what type of content they can retrieve, or what actions they can perform, as well as the appropriate links to do so.

This in page guidance via links means that your clients do not need to remember much, they can just request a resource and check the response to see how to work with information provided, take appropriate actions, or access related information.

A good example of this is a client reading your site news only needs a single endpoint https://<site>.com/news.

The response would included all of the related articles and actions which you can change daily without coupling the client to news articles in any way.

Hypermedia can be express as links in a JSON API response:

{
  "data": [{
    "type": "clients",
    "id": "1",
    "attributes": {
      "name": "John"
    },
    "links": {
      "self": "http://example.com/clients/1"
	  "accounts": "http://example.com/clients/1/accounts"	
    },
    "relationships": {
      "accounts": {
        "links": {
            { 
               "name"  : "self",
               "method": "get",
               "href"  : "http://example.com/client/1/relationships/accounts"
            },
            { 
               "name"  : "related",
               "method": "get",
               "href"  : "http://example.com/client/1/accounts"
            },
            { 
               "name"  : "next",
               "method": "get",
               "href"  : "http://example.com/client/1/accounts"
            },
            { 
               "name"  : "prev",
               "method": "get",
               "href"  : "http://example.com/client/1/accounts"
            },
            { 
               "name"  : "close",
               "method": "patch",
               "href"  : "http://example.com/client/1/accounts"
            },
          "next" : "http://example.com/clients/1/accounts?page=3&page_size=2",
          "prev" : "http://example.com/clients/1/accounts?page=1&page_size=2",
          "close": "http://example.com/clients/1/accounts/close"
        },
        "data": [
          { "type": "accounts", "id": "5" },
          { "type": "accounts", "id": "12" },
          { "type": "accounts", "id": "13" }
        ]
      }
    }
  }],
  "included": [{
    "type": "acounts",
    "id": "12",
    "attributes": {
      "type": "Investment"
    }
    "links": {
        { 
           "name"  : "close",
           "method": "patch",
           "href"  : "http://example.com/accounts/12"
        },
        { 
           "name"  : "update",
           "method": "patch",
           "href"  : "http://example.com/accounts/12"
        },
    }
  }]
}

The action links description the content that can be retrieved and the actions that can be performed by user in a response to their request.

This is powerful as it gives the server flexibility to change without breaking the interface with the client.

Use the Accept and Content-Type Headers

We tend to think, “I’m going to build a JSON REST API and it’s going to be awesome.” It works great, until you get that million?dollar customer who needs XML. Then you have to go back and refactor the entire API for this customer. That’s why you should build your API from the start with the ability to add content types in the future.

Give yourself the ability to support multiple specifications without worrying about breaking backward compatibility.

Incoming request may have an entity attached to it. To determine it’s type, server uses the HTTP request header Content-Type. Common content types are:

  • application/json
  • application/xml
  • text/plain
  • text/html
  • image/gif
  • image/jpeg

Similarly, to determine what type of representation is desired at client side, HTTP header ACCEPT is used. It will have one of the values as mentioned for Content-Type above.

Sparse Fields

Sparse fields are key to creating Api’s that can be used by many clients.

If we do not support sparse fields we will force all clients to get the full set of data which will grow we add more functionality, similar to large objects graphs created in our monolith applications.

Use a fieldsTYPE parameter to return only specific fields in the response on a per-type basis.

GET /articles?include=author&fields[articles]=title,body 

Here we want articles objects to have fields title, body and author only.

HTTP/1.1 200 OK
Content-Type: application/vnd.api+json

{
  "data": [{
    "type": "articles",
    "id": "1",
    "attributes": {
      "title": "JSON:API paints my bikeshed!",
      "body": "The shortest article. Ever."
    },
    "relationships": {
      "author": {
        "data": {"id": "42", "type": "people"}
      }
    }
  }]
}

Timeouts

The client is in a better position to tell the server how long it wants to wait before timing out, so generally allow the client to override the default timeout period.

?Timeout=3000

Caching

ETag (entity tag) response header provides a mechanism to cache unchanged resources.

It’s value is an identifier which represents a specific version of the resource. Here’s an example ETag header:

ETag: "version1"

See ETags

Warning

Designing an API – that’s the most difficult part. That’s why you need to spend your time there and say, “Let’s get the design right, and everything else will follow.”

It only takes one tiny little thing, just one mistake in your API that goes to production, to screw things up.

Just like Facebook: they have this issue in production, but it’s in production now and they can’t change it.

Back Burner

  • Filtering, Sorting & Grouping
  • Descriptive Error Messages
  • Automate end-to-end functional testing
  • Cross Origin Resource Sharing.
  • Accelerate functional testing from your CI/CD pipelines
  • Tests to generate realistic load scenarios and security attacks
  • Remove dependencies during testing and development
  • Auto-gen & publish documentation.

Links

Flutter – Layout

This is a good place to start, you can really get going with making your ideas reality, once you can show content and navigate around.

Login, logging and other framework items, although vital can and should wait until later. It’s all too easy to start a project and never get past the login screen, so simple don’t start with them.

I chose to use tab navigation for DigestableMe.

The advantage of the Tab Bar is it is always visible, one click navigation that people with Apple and Android phones are used to, which will give your app a native feel, and good Ux experience.

When you have a larger screen a Side Bar or Menu will provide a better experience. These layouts will be covered in other posts and if you are not concentrating on phone/mobile first you may want to skip this post.

Let’s assume we have analytics showing we should target iPhone users, so we can start by creating a traditional bottom Tab Bar.

In a future post we will then modify the application for Android phones and other devices.

Google recommends up to 5 top level icons and Apple 3-5, so we are going to set the max number of top level items to 5.

DigestableMe will start with these three tabs:

  • Interests, a list of things to digest.
  • Friends, people to digest with.
  • History, recent first list of actions.

The new widget will be called TabbedLayout and will contain a Tab Bar and Content Area.

Tabbed Layout Skeleton Design
Tabbed Layout Skeleton Design

Its purpose is to:

  • Highlight the core functions
  • Simplify the user journey

We can now add some acceptance criteria to give direction and to know when we are done:

  1. It will contain a Tab Bar
  2. The Tab Bar can contain 2-5 items
  3. The Tab Bar will be anchored to the bottom of the widget
  4. The height of the Tab Bar will be the size of the largest item
  5. Each item will comprise of a title and an icon
  6. Cupertino icons will be used on Apple devices, Material on all others.
  7. There will be a Content Area above the above the Tab Bar
  8. The Content Area will expand to fill the available height.
  9. When a Tab is pressed it displays the appropriate Content
  10. Tab Items and Content are configurable I.e. inputs.
  11. The  TabbedLayout widget will expand to fill the available space.
  12. The widget can be themed.
  13. The widget can display as a skeleton when loading.

To start I created these new files:

  • /test/widget/layout/tabbed_layout_tab_bar_test.dart
  • /lib/widget/layout/tabbed_layout.dart

I then started writing tests and code to:

  • Implement the criteria above
  • Provide a contract for the code.
  • Document the code through readable tests.

Ta Da

Tabbed Layout screenshots.
Tabbed Layout screenshots.
Tab Bar Test result
Tab Bar Test result
Content Test result
Content Test result
Tab Item Test result
Tab Item Test result

During the writing of the tests these additional files where added.

  • /lib/widget/layout/domain/tab_item.dart
  • /test/widget/layout/fixture/tabbed_layout_fixture.dart
  • /test/widget/layout/tabbed_layout_tab_item_test.dart
  • /test/widget/layout/tabbed_layout_content_test.dart

Some of the acceptance criteria have been skipped because the functionality is covered and tested by the CupertinoTabScaffold widget:

  1. The Tab Bar will be anchored to the bottom of the widget
  2. The height of the Tab Bar will be the size of the largest item
  3. The Content Area will expand to fill the available height.
  4. When a Tab is pressed it displays the appropriate Content
  5. The Tabs widget will expand to fill the available space.

See the code for more details on the new widget and tests.

BackBurner

  • Using material design for other devices and browsers, to give a cross platform look and feel.
  • Branding, themes.

Including these acceptance criteria:

  1. Cupertino icons will be used on Apple devices, Material on all others.
  2. The widget can be themed.
  3. The widget can display as a skeleton when loading.

XP

VSCode

Use the VSCode extension

VSCode extenstion - Awesome Flutter Snippets
VSCode extenstion – Awesome Flutter Snippets

to create the stateful widget skeleton code using a snippet.

Using the stateful widget snippet.
Using the stateful widget snippet.

Use the VSCode extension to create the TabItem class.

Package, Dart data class generator
Package, Dart data class generator

NB: I reduced the codeine to the constructor and toString, we will add more if needed in later posts.

People

Kent Beck, Agile Manifesto founder, TDD, extreme programming.

Picture of Kent Beck

eXtreme programming practices
eXtreme programming practices

Sound & Vision

Love my way, Psychodelic Furs.

YAGNI, just sounded like Cagney…

Film: Public Enemy - James Cagney

Links

One more thing

We had a fundamental belief that doing it right the first time was going to be easier than having to go back and fix it. And I cannot say strongly enough that the repercussions of that attitude are staggering. I’ve seen them again and again throughout my business life.

Steve Jobs_

Confusion without Fusion

Error

The binding error:

  • located assemblies manifest definition does not match
The binding error.
The binding error.

Diagnosis

Excerpt from the fusion log
Excerpt from the fusion log

Fix

Example of adding a binding redirect to fix the issue.
Example of adding a binding redirect to fix the issue.

Toolin…

I started simple thinking I would improve, and never have.

These .bat scripts and FusLogVwSet.wsf from Travis Illig, are more than enough to get the job done.

Shows the batch files used to control logging.
Shows the batch files used to control logging.

Enable Fusion

FusionLogEnable.bat

e:
cd commands
IF EXIST d:\fusionlogs RMDIR /S /Q d:\fusionlogs
cscript fuslogvwset.wsf /enable
iisreset 
pause 0 

View
c:
cd "C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.6.1 Tools\"
FUSLOGVW.exe

Disable Fusion

FusionLogDisable.bat

e:
cd commands
cscript fuslogvwset.wsf /disable 
iisreset 
pause 0

View Fusion Log

FusionLogView.bat

c:
cd "C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.6.1 Tools\"
FUSLOGVW.exe

FusLogVwSet.wsf

https://www.paraesthesia.com/archive/2004/10/20/fusion-log-viewer-settings-changer.aspx/

\<?xml version="1.0" ?\>
\<?job error="true" debug="false" ?\>
\<!--
'============================================================================
' FUSION LOG VIEWER SETTINGS
' FusLogVwSet.wsf
' Travis Illig
' tillig@paraesthesia.com
' http://www.paraesthesia.com
'
' Overview:  Enables/disables custom settings for the fuslogvw.exe tool.
'
' Command syntax:  (Run "FusLogVwSet.wsf /?" for syntax and usage)
'
'============================================================================
--\>
\<package\>
\<job id="FusLogVwSet"\>
\<runtime\>
\<description\>
FusLogVwSet
---- 
This script "enables" and "disables" custom settings for the Fusion Log Viewer tool.

Enabling settings will:
* Create a log folder (default: D:\fusionlogs)
* Add HKEY\_LOCAL\_MACHINE\SOFTWARE\Microsoft\Fusion\LogPath and set it to the log folder
* Set HKEY\_LOCAL\_MACHINE\SOFTWARE\Microsoft\Fusion\LogFailures to 1
* Optionally set HKEY\_LOCAL\_MACHINE\SOFTWARE\Microsoft\Fusion\ForceLog to 1
* Optionally set HKEY\_LOCAL\_MACHINE\SOFTWARE\Microsoft\Fusion\LogResourceBinds to 1

Disabling settings will:
* Delete the log folder and its contents
* Delete HKEY\_LOCAL\_MACHINE\SOFTWARE\Microsoft\Fusion\LogPath
* Set HKEY\_LOCAL\_MACHINE\SOFTWARE\Microsoft\Fusion\LogFailures to 0
* Set HKEY\_LOCAL\_MACHINE\SOFTWARE\Microsoft\Fusion\ForceLog to 0
* Set HKEY\_LOCAL\_MACHINE\SOFTWARE\Microsoft\Fusion\LogResourceBinds to 0
	\</description\>
	\<named
	name="enable"
	helpstring="Enable custom fuslogvw.exe settings."
	type="simple"
	required="false"
	/\>
	\<named
	name="all"
	helpstring="When used with /enable, logs both failures and successes.  Only valid with /enable."
	type="simple"
	required="false"
	/\>
	\<named
	name="disable"
	helpstring="Disable custom fuslogvw.exe settings."
	type="simple"
	required="false"
	/\>
	\<named
	name="logpath"
	helpstring="Sets the log path (default is D:\fusionlogs).  Only valid with /enable."
	type="string"
	required="false"
	/\>
	\</runtime\>
	
	
	\<!-- Helper Objects --\>
	\<object id="fso" progid="Scripting.FileSystemObject" /\>
	\<object id="shell" progid="WScript.Shell" /\>
	
	\<!-- Main Script --\>
	\<script language="VBScript"\>
	\<!\[CDATA\[
	]()
	'============================================================================
	' INITIALIZATION
	Option Explicit
	
	'Declare variables/constants
	Const SCRIPTNAME = "Fusion Log Viewer Settings"
	Const VERSION = "1.0"
	
	Const DEFAULT\_FUSIONLOGPATH = "D:\fusionlogs"
	Const REG\_LOGPATH = "HKLM\SOFTWARE\Microsoft\Fusion\LogPath"
	Const REG\_LOGFAILURES = "HKLM\SOFTWARE\Microsoft\Fusion\LogFailures"
	Const REG\_FORCELOG = "HKLM\SOFTWARE\Microsoft\Fusion\ForceLog"
	Const REG\_RESOURCEBINDS = "HKLM\SOFTWARE\Microsoft\Fusion\LogResourceBinds"
	
	
	'============================================================================
	'PRIMARY CODE
	'============================================================================
	On Error Resume Next
	
	WScript.echo SCRIPTNAME & " v" & VERSION & vbCrLf
	
	'Parse arguments
	Dim argsSpecified
	Dim argsEnable, argsDisable, argsAll, argsLogPath
	
	argsEnable = WScript.Arguments.Named.Exists("enable")
	argsDisable = WScript.Arguments.Named.Exists("disable")
	argsAll = WScript.Arguments.Named.Exists("all")
	If(WScript.Arguments.Named.Exists("logpath"))Then
	argsLogPath = WScript.Arguments.Named.Item("logpath")
	End If
	
	'Validate arguments
	If(not argsEnable and not argsDisable)Then
	' Must specify either enable or disable
	WScript.Echo "\*\*\* You must specify enable or disable."
	WScript.Arguments.ShowUsage
	WScript.Quit
	End If
	
	If(argsEnable and argsDisable)Then
	' Can't enable and disable at the same time
	WScript.Echo "\*\*\* You must specify EITHER enable OR disable; not both."
	WScript.Arguments.ShowUsage
	WScript.Quit
	End If
	
	If(argsDisable and argsAll)Then
	'all is only valid with enable
	WScript.Echo "\*\*\* Argument 'all' is only valid with 'enable'."
	WScript.Arguments.ShowUsage
	WScript.Quit
	End If
	
	If(argsDisable and WScript.Arguments.Named.Exists("logpath"))Then
	'logpath is only valid with enable
	WScript.Echo "\*\*\* Argument 'logpath' is only valid with 'enable'."
	WScript.Arguments.ShowUsage
	WScript.Quit
	End If
	
	If(argsLogPath = "" and WScript.Arguments.Named.Exists("logpath"))Then
	'If logpath is specified, must put a value
	WScript.Echo "\*\*\* Argument 'logpath' must have a value if specified."
	WScript.Arguments.ShowUsage
	WScript.Quit
	End If
	
	
	' Output settings
	If(argsEnable)Then
	If(argsAll)Then
	LogMessage "Action: Enable Custom Logging - Failure and Success", 0
	Else
	LogMessage "Action: Enable Custom Logging - Failure Only", 0
	End If
	If(argsLogPath \<\> "")Then
	LogMessage "LogPath: " & argsLogPath, 0
	End If
	Else
	LogMessage "Action: Disable Custom Logging", 0
	End If
	
	
	' Update settings
	Dim logFolder, logFolderObj, regVal
	If(argsEnable)Then
	' Enable settings
	' Create a log folder (default: D:\fusionlogs)
	If(argsLogPath = "")Then
	logFolder = DEFAULT\_FUSIONLOGPATH
	Else
	logFolder = argsLogPath
	End If
	
	If(FolderExists(logFolder))Then
	' The folder already exists; since we're deleting it when we disable
	' settings, we don't want to use a pre-existing folder.
	LogMessage "Folder " & logFolder & " exists.  Custom log folder must not already exist.", 1
	WScript.Quit(0)
	End If
	
	Set logFolderObj = fso.CreateFolder(logFolder)
	If Err.Number \<\> 0 Then
	LogMessage "Unable to create log folder" & logFolder, 1
	WScript.Quit(-1)
	End If
	Err.Clear
	LogMessage "Created log folder " & logFolderObj.Path, 0
	
	
	' Add HKEY\_LOCAL\_MACHINE\SOFTWARE\Microsoft\Fusion\LogPath and set it to the log folder path.
	SetRegKey REG\_LOGPATH, logFolderObj.Path, "REG\_SZ"
	regVal = GetRegKey(REG\_LOGPATH)
	If(regVal \<\> logFolderObj.Path)Then
	LogMessage "Unable to write registry key " & REG\_LOGPATH, 1
	WScript.Quit(-1)
	End If
	LogMessage "Wrote to " & REG\_LOGPATH, 0
	
	' Set HKEY\_LOCAL\_MACHINE\SOFTWARE\Microsoft\Fusion\LogFailures to 1
	SetRegKey REG\_LOGFAILURES, 1, "REG\_DWORD"
	regVal = GetRegKey(REG\_LOGFAILURES)
	If(regVal \<\> 1)Then
	LogMessage "Unable to write registry key " & REG\_LOGFAILURES, 1
	WScript.Quit(-1)
	End If
	LogMessage "Wrote to " & REG\_LOGFAILURES, 0
	
	If(argsAll)Then
	' Optionally set HKEY\_LOCAL\_MACHINE\SOFTWARE\Microsoft\Fusion\ForceLog to 1
	SetRegKey REG\_FORCELOG, 1, "REG\_DWORD"
	regVal = GetRegKey(REG\_FORCELOG)
	If(regVal \<\> 1)Then
	LogMessage "Unable to write registry key " & REG\_FORCELOG, 1
	WScript.Quit(-1)
	End If
	LogMessage "Wrote to " & REG\_FORCELOG, 0
	
	' Optionally set HKEY\_LOCAL\_MACHINE\SOFTWARE\Microsoft\Fusion\LogResourceBinds to 1
	SetRegKey REG\_RESOURCEBINDS, 1, "REG\_DWORD"
	regVal = GetRegKey(REG\_RESOURCEBINDS)
	If(regVal \<\> 1)Then
	LogMessage "Unable to write registry key " & REG\_RESOURCEBINDS, 1
	WScript.Quit(-1)
	End If
	LogMessage "Wrote to " & REG\_RESOURCEBINDS, 0
	End If
	Else
	' Disable settings
	logFolder = GetRegKey(REG\_LOGPATH)
	If(logFolder = "")Then
	LogMessage "Unable to read registry key " & REG\_LOGPATH, 1
	WScript.Quit(-1)
	End If
	
	If(FolderExists(logFolder))Then
	' The folder exists; delete it and its contents
	fso.DeleteFolder logFolder, true
	If Err.Number \<\> 0 Then
	LogMessage "Unable to delete log folder" & logFolder, 1
	WScript.Quit(-1)
	End If
	Err.Clear
	LogMessage "Deleted log folder " & logFolder, 0
	End If
	
	' Delete HKEY\_LOCAL\_MACHINE\SOFTWARE\Microsoft\Fusion\LogPath
	If(DeleteRegKey(REG\_LOGPATH))Then
	LogMessage "Deleted registry key " & REG\_LOGPATH, 0
	Else
	LogMessage "Unable to delete registry key " & REG\_LOGPATH, 1
	WScript.Quit(-1)
	End If
	
	' Set HKEY\_LOCAL\_MACHINE\SOFTWARE\Microsoft\Fusion\LogFailures to 0
	SetRegKey REG\_LOGFAILURES, 0, "REG\_DWORD"
	regVal = GetRegKey(REG\_LOGFAILURES)
	If(regVal \<\> 0)Then
	LogMessage "Unable to write registry key " & REG\_LOGFAILURES, 1
	WScript.Quit(-1)
	End If
	LogMessage "Wrote to " & REG\_LOGFAILURES, 0
	
	' Set HKEY\_LOCAL\_MACHINE\SOFTWARE\Microsoft\Fusion\ForceLog to 0
	SetRegKey REG\_FORCELOG, 0, "REG\_DWORD"
	regVal = GetRegKey(REG\_FORCELOG)
	If(regVal \<\> 0)Then
	LogMessage "Unable to write registry key " & REG\_FORCELOG, 1
	WScript.Quit(-1)
	End If
	LogMessage "Wrote to " & REG\_FORCELOG, 0
	
	' Set HKEY\_LOCAL\_MACHINE\SOFTWARE\Microsoft\Fusion\LogResourceBinds to 0
	SetRegKey REG\_RESOURCEBINDS, 0, "REG\_DWORD"
	regVal = GetRegKey(REG\_RESOURCEBINDS)
	If(regVal \<\> 0)Then
	LogMessage "Unable to write registry key " & REG\_RESOURCEBINDS, 1
	WScript.Quit(-1)
	End If
	LogMessage "Wrote to " & REG\_RESOURCEBINDS, 0
	End If
	
	LogMessage "Log settings update COMPLETE.  You must reset IIS for changes to take effect in ASP.NET apps.", 0
	
	On Error Goto 0
	Wscript.Quit(0)
	
	
	'============================================================================
	' CreateNewObject
	'
	' Creates a new object, given a type, and performs requisite error checking.
	' Exits the program if the object can't be created.
	'
	Function CreateNewObject(objType)
	On Error Resume Next
	
	'Create a new object
	Dim obj
	Set obj = WScript.CreateObject(objType)
	If Err.Number \<\> 0 Then
	LogMessage "Unable to create " & objType, 1
	WScript.Quit(-1)
	End If
	Err.Clear
	
	Set CreateNewObject = obj
	
	On Error Goto 0
	End Function
	
	
	'============================================================================
	' FolderExists
	'
	' Returns a Boolean based on whether a folder exists or not
	'
	Function FolderExists(foldername)
	On Error Resume Next
	
	'Create a FileSystemObject object
	Dim fso
	Set fso = CreateNewObject("Scripting.FileSystemObject")
	
	'Check for the folder
	FolderExists = false
	FolderExists = fso.FolderExists(foldername)
	
	Set fso = Nothing
	
	On Error Goto 0
	End Function
	
	
	'============================================================================
	' DeleteRegKey
	'
	' Deletes a given registry key
	' Returns true if the delete was successful, false otherwise
	'
	Function DeleteRegKey(regkey\_name)
	On Error Resume Next
	
	'Create a shell object
	Dim wshell
	Set wshell = CreateNewObject("WScript.Shell")
	
	'Write the regkey
	wshell.RegDelete regkey\_name
	If Err.Number \<\> 0 Then
	'Something else went wrong
	LogMessage "Unable to delete key " & regkey\_name, 1
	DeleteRegKey = false
	Else
	DeleteRegKey = true
	End If
	Err.Clear
	
	Set wshell = Nothing
	
	On Error Goto 0
	End Function
	
	
	'============================================================================
	' SetRegKey
	'
	' Sets the value for a given registry key
	'
	Sub SetRegKey(regkey\_name, regkey\_value, regkey\_type)
	On Error Resume Next
	
	'Create a shell object
	Dim wshell
	Set wshell = CreateNewObject("WScript.Shell")
	
	'Write the regkey
	wshell.RegWrite regkey\_name, regkey\_value, regkey\_type
	If Err.Number \<\> 0 Then
	'Something else went wrong
	LogMessage "Unable to write key " & regkey\_name, 1
	End If
	Err.Clear
	
	Set wshell = Nothing
	
	On Error Goto 0
	End Sub
	
	
	'============================================================================
	' GetRegKey
	'
	' Retrieves the value for a given registry key
	'
	Function GetRegKey(regkey\_name)
	On Error Resume Next
	
	'Create a shell object
	Dim wshell
	Set wshell = CreateNewObject("WScript.Shell")
	
	'Read the regkey
	Dim val
	val = wshell.RegRead(regkey\_name)
	If Err.Number \<\> 0 Then
	'Either we don't have permission to read the key or the key doesn't exist.
	' If the key doesn't exist, it's error -2147024894
	If Err.Number = -2147024894 Then
	'The key doesn't exist
	val=""
	Else
	'Something else went wrong
	LogMessage "Unable to read key " & regkey\_name, 1
	val=""
	End If
	End If
	Err.Clear
	
	Set wshell = Nothing
	
	GetRegKey = val
	On Error Goto 0
	End Function
	
	
	'============================================================================
	' LogMessage
	'
	' Writes a message to the event log
	'
	' msgType:
	'   0 = Info
	'   1 = Error
	'   2 = Warning
	Sub LogMessage(msgBody, msgType)
	On Error Resume Next
	
	'Create a shell object
	Dim wshell
	Set wshell = WScript.CreateObject("WScript.Shell")
	If Err.Number \<\> 0 Then
	WScript.Quit(-1)
	End If
	Err.Clear
	
	'Figure out the error type
	Dim msgTypeFull
	If(msgType = 0) Then
	msgTypeFull = "INFO"
	ElseIf(msgType = 1) Then
	msgTypeFull = "ERROR"
	ElseIf(msgType = 2) Then
	msgTypeFull = "WARNING"
	End If
	
	msgBody = WScript.ScriptName & " -- " & msgTypeFull & ": " & msgBody
	wscript.echo msgbody
	
	'Log the message
	wshell.LogEvent msgType, msgBody
	
	'Cleanup
	Set wshell = Nothing
	
	On Error Goto 0
	End Sub
	
	]]\>
	</script\>
	\</job\>
</package\>

Flutter – CI Web

Web was put on the back burner in the previous post ‘Flutter – CI Workflow’.

In this post we are going to look at the steps to deploy a website to Firestore and discuss other hosts.

Get and add a firebase token to your environment variables by following Codemagic guide to ‘Publishing web application to Firebase Hosting

Shows the FIREBASE_TOKEN environment variable
Shows the FIREBASE_TOKEN environment variable

Next, add a couple of tasks to the codemagic.yaml.

Code snippet showing a CI build task to build a website in a codemagic.yaml workflow
Code snippet showing a CI build task to build a website in a codemagic.yaml workflow
Code snippet showing a CI build task to deploy a website to firebase in a codemagic.yaml workflow
Code snippet showing a CI build task to deploy a website to firebase in a codemagic.yaml workflow

Ta Da

Hosted for UAT exploratory testing.

Codemagic build results showing just the web build and deployment tasks
Codemagic build results showing just the web build and deployment tasks
Build results show the website deployment to Firebase
Build results show the website deployment to Firebase

Full Integration workflow.

Results from the codemagic integration build.
Results from the codemagic integration build.

Other Cloud Services

Codemagic can easily integrate with other cloud services, if the build agents are missing service CLI’s you can use the pip command to add them to the agent as part of the build and then run the commands needed to deploy the build.

AWS

Just follow the guide ‘CI/CD with Flutter, Codemagic, and AWS S3’ on the Codemagic site.

Codemagic.yaml task to deploy to AWS

Azure

Show an overview of the process to deploy a website from Codemagic to Azure static web pages.

You can install the Azure CLI as part of your build workflow, then issue commands to login and deploy the website build files.

pip install azure-cli 

az login --service-principal -u <app-id> -p $AZURE_CERT --tenant $AZURE_TENANT

az storage blob upload-batch --account-name simbuapp123 -s ./builds/web -d '$AZURE_WEBSITE_ID'
  • The URL or name associated with the service principal
  • The service principal password, or the X509 certificate used to create the service principal in PEM format
  • The tenant associated with the service principal, as either an .onmicrosoft.com domain or Azure object ID

BackBurner

Shows that BrowserStack integrates with codemagic

Browserstack integration will be covered in another post.

It will look at how to use it for both multiple browser and multiple device testing.

XP

Just one thing of importance to note this time, the website gets built to the ‘build\web’ directory on the agent.

Shows the path that the website build is saved to.
Shows the path that the website build is saved to.

Sound & Vision

Link to Adele's new album 30

Links

Flutter – CI Workflow

I didn’t expect that this would be the first blog post in the series!

In order to get Flutter accepted and approved at my company, I’ve been asked to demonstrate the end process of testing and deploying a flutter application.

Feel free to come back to this blog post when you’re closer to shipping your shiny new Flutter application.

TL;DR; Code: codemagic.yaml Read From: Setting up the CI workflow

Focus and the amount of Friction are two key things I look at when thinking about design, workflows and changes to them.

There is a tendency to only care about deployment and environments when they don’t work, and to make do for as long as possible.

This can be very costly as they can easily become 80% of your development effort if you make it difficult, overly bespoke or include numerous manual steps.

You won’t regret the effort or cost to set up a good CI/CD workflow, using the best available tools, you can then get back to building great Flutter applications.

I started by reviewing the tools recommend by Flutter for continuous integration and delivery.

I choose GitLab, having had some experience of it before, for its powerful yaml configuration, large feature set and great integrations.

It’s an all in one DevOps platform delivered as a single application, you can move really fast with GitLab.

CodeMagic has great built in support for Flutter, is highly configurable and has a set of virtual Mac’s that allows you to build & sign applications for Apple devices and the App store.

It makes the whole process much simpler and integrates well with GitLab.

Git is great, Git is powerful, but it will let you create an indecipherable mess when working with a team on many projects and features.

If it’s just you, check into one branch, usually called master or main and deploy from it. Super simple, on you go.

With a team/s there are many features, projects, people and releases that need coordinated and controlled, you will require a branch structure and a process around it.

Lots of teams are successfully using Gitflow, but it takes a little time to get developers rowing in the same direction, requires lots of merges and makes the history tricky to understand.

I wanted something simple like the single branch that would work for teams and I’ve chosen the Streamline git workflow.

If it causing friction we can adjust and if it morphs back to Gitflow in the end so be it, we have made the leap in understanding.

So now we have the tool chain to build the workflows:

The continuous integration workflow will be triggered when developers check in to any feature branch with a tag beginning “CI”.

Future posts will cover the continuous deployment workflow, automated versioning and integration and deployment for a website version.

Toolset Cost Justification

GitLab & CodeMagic

Costs and justification for using Codemagic with Gitlab.
Costs and justification for using Codemagic with Gitlab.

Gitlab is single application that covers the total development and operations cycle and can run advanced security tests on the software you write.

It integrates easily with most other frameworks and services allowing you to move fast and create a fully automated system in no time.

CodeMagic targets mobile integration and deployment and brings some powerful tools and features to the table:

  • API’s to communicate with Apple and Google, so you can sign and build apps automatically.
  • A farm of Macs to allow you to build for iOS and macOS from any device.
  • Strong integration with Google cloud to give you access to the Firestore services.
  • GitLab and CodeMagic integrate seamlessly together.

Whilst you could manually create servers and scripts to cover the tasks these services provide and hook them up in other CI/CD services like TFS or Jenkins, you would have to maintain them along with a number of virtual Mac agents and without support.

Even if you managed to find one person to cover this work at a modest salary of £20k per annum that would be a much greater cost.

Setting up the CI workflow

Ok enough talk lets go!

Show an overview of the main CI/CD tasks.

I started out by writing the list of actions I wanted the Integration workflow to carry out:

  • Check code quality.
  • Run Unit and Widget tests.
  • Sign and build an iOS version of the app.
  • Sign and build an Android version of the app.
  • Run Integration tests using Google’s Firebase TestLab.
  • Send an install on device link to testers when a build succeeds.

All of these tasks are setup in the codemagic.yaml file that you add to the root of your flutter process and configured with the Integration and Deployment tasks that you require.

The structure of the file is:

  • We will create the CI workflow under my-workflows.
  • Use the environment section to import secret values to use API’s to search and control external services including the App Store, Play Store and Firebase.
  • Add script tasks under scripts to build and sign the application for each platform required.
  • Configure artifacts to make the outputs available when the build completes.
  • Add recipients under email to send an email with app install links out to the testers

Setup – Service Access

In order to setup the workflow we will need access to Cloudmagic, the App Store, Play Store and Firebase services so we can use their API’s.

You will need to have or setup the service accounts before continuing, the apple developer account costs £79 per year, the google developer account is a $25 registration fee and Firebase and Cloudmagic have a free tier to get you started.

Follow the instructions in the section Service Access below, and gather the secrets:

appstore_credentials

APP_STORE_CONNECT_ISSUER_ID
APP_STORE_CONNECT_KEY_IDENTIFIER
APP_STORE_CONNECT_PRIVATE_KEY 
CERTIFICATE_PRIVATE_KEY
BUNDLE_ID

firebase_credentials

GCLOUD_KEY_FILE
FIREBASE_PROJECT

I stored the secret info and corresponding Environment Vars Names in secure notes in my Apple account Keychain, its up to you but I would recommend you keep them safe and secure.

Setup – Adding secrets as Environment Variables

Use the Codemagic UI to easily create the group and secure environment variables:

Then include the groups in the environment section of the codemagic.yaml workflow:

Checkout the documentation for more info on common Environment Variables:

We can now start adding the tasks tasks to our Codemagic integration workflow.

You can access the the workflow code anytime in the DigestableMe project on GitLab in the codemagic.yaml file.

Task – Check Code Quality

This task will run code analysis on the project and highlight code that will be difficult to maintain or puts the codebase at risk.

The task is added to the scripts section in codebase.yaml

The rules are add to the project pubspec.yaml file.

The output is save as a build artifact:

Task – Run local unit and widget tests

This task will pick up any unit or widget tests files under the /test directory of the project that end in _test.dart

Task script:

Artifact:

Task – Sign and build an iOS version of the application

Script Tasks:

Artifact:

Codemagics API to integrate with Apple really helps you out with the signing process.

It will work out what the app needs and create any certificates to complete the signing, amazing…

Task – Sign and build an Android version of the application

Script Tasks:

Artifact:

Task – TestLab

With the Google’s Firebase TestLab you can run your coded integration tests on multiple real devices and run a robo test that will discover and run through all the screens in your app, again on multiple devices.

Powerful stuff that will give you confidence that you application can run on devices that you wish to support.

The flip side is these valuable tests take time to run and that comes with a £ cost for using the service.

To setup integrations tests and get your project ready for Firebase TestLab run through the Flutter Integration Test Tutorial.

Once setup add the two scripts to the integration workflow:

See the sections under XP below for details on how to run TestLab locally.

Ta Da

Built and Regression Tested!

When everything passes and the testers are notified by email with links to install the version locally on their devices!

Service Access

App Store Connect Access

See the Codemagic instructions for more details.

NB: You may need to refresh the page after creation before the download link becomes visible.

Collect the secret values:

APP_STORE_CONNECT_ISSUER_ID
APP_STORE_CONNECT_KEY_IDENTIFIER
APP_STORE_CONNECT_PRIVATE_KEY 
CERTIFICATE_PRIVATE_KEY
BUNDLE_ID

The issuer id and key identifier are values you saved during the creation of the API key.

The ‘APP_STORE_CONNECT_PRIVATE_KEY’ is the key you download from the App Store Connect after creating the API key, the <hash>.p8 file. Just copy the contents directly into the environment variable value.

The ‘CERTIFICATE_PRIVATE_KEY’ is an RSA 2048 bit private key to be included in the signing certificate that Codemagic creates. You can use an existing key or create a new 2048 bit RSA key:

ssh-keygen -t rsa -b 2048 -m PEM -f ~/Desktop/codemagic_private_key -q -N ""

Copy the all the content from the private key file into the environment variable value.

BUNDLE_ID is the unique id for application and you define it when you create an identity in your Apple developer account.

You can find the iOS bundle id in your apple developer account under identifiers.

Apple recommend using a reverse-domain name style string i.e.

icom.domainname.appname

It cannot contain an asterisk.

Firebase Access for TestLab

You will need to set up the following in the Firebase console.

Collect the secret values:

GCLOUD_KEY_FILE
FIREBASE_PROJECT

Where:

GCLOUD_KEY_FILE – service account JSON key file, FIREBASE_PROJECT – your Firebase Project ID, you can find it under project-settings-general in the Firebase console.

Play Store Access & Signing

Google allows you to manage your signing credentials locally, but it’s better to let Google manage them in the cloud like the Apple App Store does.

For this workflow we will just create a default debug keystore directly in the codebase.yaml file, which will be fine for integration and ad-hoc testing.

When we need to deploy to the Play Store we will setup the keystore in the Google cloud, this will be covered in the continuous deployment post.

XP

Codemagic

I found most difficult thing at first was to understand the sections in codebase.yaml file.

In addition to the official documentation and google, a few of things really helped with this.

Firstly you can setup workflow using the UI and then switch to yaml configuration and export the values from the workflow you have setup.

You can use builder mode to give some contextual help.

When things go a little tougher like integrating with TestLab I eventually had to download the Firebase CLI and get the script running locally before going back to the workflow and plugging it in.

Firebase

I needed to install the API to list device models for TestLab and run tests without going through CodeMagic that was costing a lot of build minutes.

See Firebase CLI reference  for more details. On the Mac you can install it with the this command.

curl -sL https://firebase.tools | bash

Then login with this command.

firebase login

Install gcloud

curl https://sdk.cloud.google.com | bash 

Login

gcloud auth login

And list device models with this.

gcloud firebase test android models list 

Note you can change the gcloud login account(email) with

gcloud config set account ACCOUNT

Firebase TestLab

On the free spark plan you have the following allowance

Once you have the firebase and gcloud CLI’s installed you can run TestLab tests directly and plug the commands back into the codemagic.yaml file.

To run the tests you will need to set the project id first:

gcloud config set project PROJECT_ID

You can find the project id by list all projects in your firebase console.

Do this by downloading the Android build artifacts and then modifying the codemagic.yaml command from

To

gcloud firebase test android run \
          --type instrumentation \
          --app app-debug.apk \
          --test app-debug-androidTest.apk \
          --timeout 3m

So that it points at the download build artifacts.

You can specify a device using —device, can be added multiple times for multiple devices.

          --device model=redfin,orientation=portrait  \

Be careful to choose a device that supports your SDK, when devices have multiple then you need to specify one:

          --device model=redfin,version=28  \

See gcloud firebase test android run command documentation for more command options.

Once you have the tests setup and passing make the changes back to codemagic.yaml and you are good to go.

BackBurner

Local Integration Tests

If you run up an emulator or attach a device then you can run all the integrations tests locally.

I would have liked to run local integration tests before the TestLab tests to catch breaking tests earlier and avoid running breaking tests on multiple devices, which is costly, time and money.

You can run the tests local using this command, but you need a device or emulator running.

flutter drive --driver=test_driver/integration_driver.dart --target=integration_test/app_test.dart 

Web Integration

I ran out of time to add the following actions to my CI workflow.

  • Build a website version.
  • Run integration tests BrowserStack or other test tool.

Google Play Store – Automatic cloud managed signing

Implement  automatic cloud managed signing as described in the Flutter documentation to sign an Android app.

For now I’ve just created a debug version of the keystore, directly in the codemagic.yaml file.

Integrate iOS app with Firebase Test Lab.

The official documentation requires a manual step to build the iOS runner for integration and upload to Firebase as a zipped file.

For now I’ve decided to integrate the Android package and come back to this.

Ideally I will be able to create a script to package the iOS application for Firebase TestLab.

One of the options I will consider when returning to this is integrate with FastLane and use the plugin to integrate with Firebase TestLabs

Fastlane plugin – Firebase TestLab

Sound & Vision

Links

One more thing…

Generally, the best way to learn git is probably to first only do very basic things and not even look at some of the things you can do until you are familiar and confident about the basics.

_Linus Torvalds

Flutter – Lets Go

Series links

iOS Development:

  1. Layout
  2. Lists
  3. Adding Items
  4. Themes and Branding
  5. Scrolling Content
  6. Adaptive Layout
  7. Split view for the iPad 
  8. iPad side Menu
  9. Android Layout

Development

  1. Tests
  2. Packages
  3. Widgetbook (Storybook)
  4. Widgetbook (Storybook) Part2
  5. Lottie Animation

DevOps:

  1. Continuous Integration – Workflow
  2. Continuous Integration – Web

About

Flutter is exciting, there is a lot out there and I’m glad to be back building UI again, for mobiles and the web.

Join me on a series of blog posts while I learn and explore this great new cross platform framework.

My example project is called DigestibleMe and is a sharable collection of things of interest, music, food, travel, sport.

You can find the accompanying code on GitLab.

It’s important to focus on the functionality, to see your ideas come to life and not get bogged down in logos, login etc.. that can be tackled later.

The blog posts focuses on functionality needed to build DigestableMe so we will start with Layout and then add topics as needed to build the application.

I would recommend that you bookmark this post as it contains an index of articles in the series and links to design concepts.

VSCode

I have chosen to use Visual Studio Code to build DigestableMe.

Here is a list of useful utilities and shortcuts I have used on this project.

Shortcuts

Show method contextual help, options.

Ctrl + Space

Format document.

Ctrl + ? + F

Refactor, good for wrapping widgets with container.

Ctrl + ? + R

Layout

Each blog post will have the following layout and sections will be included if required:

Ta Da

Demonstration of what’s been created.

XP

Useful experiences and insights based on using things.

You may get various build errors when running your application from vscode via launch.json.

If you do then running it from the flutter cli at the command line should fix the error, force a full rebuild.

BackBurner

Things to come back to later.

Links

Links to things I’ve googled, read, viewed, come across whilst writing the blog and any useful resources to dig deeper.

People

More links, quotes, info on interesting people like Uncle Bob.

Sound and Vision

What I’m listening to or watching.

One more thing…

Quotes mainly Steve Jobs, but this will change.