Milvus milvus -Laurieston, Dumfries and Galloway, Scotland -feeding station-8-cropped

Deploying EKS with Prometheus and Grafana

Introduction

On the face of it this is a fairly pedestrian post subject. The devil, of course, is in the details. Here I wanted to deploy a new EKS cluster, with Prometheus and Grafana, cleanly and with good architectural domain boundaries. I am using the open source self-hosted stack, not the managed versions in this exercise. I also deployed using Open Tofu.

The code and the short version is HERE. The following is the commentary.

Read more →

Extrapolating the curve for targetted consumer AI services- focus Arc Search

This evening one of the the big conversation pieces on Tech Twitter is Arc Search and their ‘Act II of the internet’ launch video:

“Our vision for Act II of Arc Internet pic.twitter.com/1RidKJVYUX” - Josh Miller February 1, 2024

Commenters are talking about what this means for Google and their ’ten blue links’ but I think there are some lessons from history that might be worth considering here and some examples that may give us an idea of what to expect next.

Read more →

Going Bananas For The Minions Game

Recently I attempted a Python challenge on HackerRank - ‘The Minion Game’. My curiosity was piqued and I wanted to find out if the odds were as stacked as I initially thought. I was surprised that this wasn’t already established publicly and so I worked it out ‘myself’.

The Game

The initial task was to complete code to calculate Kevin and Stuart’s scores programmatically for this game (I believe this description is free to reproduce)- via ChatGPT:

Read more →

Writing my own (Python) study course with ChatGPT - part 2

This is a follow-up article to one I published in June 2023 - ‘Writing my own (Python) study course with ChatGPT - part 1’. This discusses a more structured and disciplined approach with reflective practice and constructive critique as part of learning and improvement.

What changed?

I had been searching for, sometimes random, exercises. Whilst discussing the first article (above) during a job interview, an interviewer suggested that Code Golf might be a good alternative to the sometimes complex and advanced challenges given in ‘Advent of Code’. I would have to agree that many of the Code Golf challenges are a better fit for cultivating familiarity with decomposing problems, language syntax etc. They are certainly much more concise in what is required and generally quick enough to tackle in a relatively short time. The brevity is a key strength, because if it’s something that is going to take hours and hours then it will be something that is only tackled occasionally or rarely. With shorter exercises it is possible to tackle something most days. The other great thing wth Code Golf is that it validates solutions as either passing or not.

Read more →

Terraform, OpenTF, and the Shrinking Gap

introduction

Hashicorp recently made shock licensing changes to their stack to close their source code to ‘competitors’. There has been much public comment and Open Terraform has already been announced in response. I see a strong future for OpenTF and a move towards Terraform itself being less used directly and more as an intermediate layer or stage for higher level tools as the gap that it occupies shrinks, as detailed below.

Read more →

AI Assistance On The Home Turf

I’ve written previously about using ChatGPT to provide Artificial Intelligence (‘AI’) support for coding and tutoring in unfamiliar areas. What about using it on home turf, i.e. in an area with which I am familiar- in this case a Terraform module for generic use?

(W)here’s the Code

It’s not necessary for following the below article but the code is published on GitHub and in the Community module registry

Background

Recently I was tasked, in a consultancy capacity, with providing a Terraform module for a (generic) AWS IAM Role for private/team use. It was up to me whether I wrote one from scratch or used public code. There are something like (approaching) 200 modules for this in the Hashicorp public community module registry so I reviewed some of the top candidates there. This is a requirement that has existed for a long time, where there have been various changes to recommended practice over time, new features introduced and some, whilst still supported, are deprecated. Notably it’s also a space where there aren’t necessarily clear boundaries to the problem and different publishers have provided solutions with different feature sets, e.g. some modules feature creating aws_iam_openid_connect_provider or aws_iam_instance_profile. In my view, whilst closely associated, these aren’t part of an IAM role. I didn’t have time to review every offering and so I created a new one for our team. Due to pressures of time and restrictions on tooling there were some corner features that I did not get to develop fully before we moved on, although they were not needed for the use case to that point.

Read more →

Writing my own (Python) study course with ChatGPT - part 1

Introduction

I’ve previously written about writing applications with ChatGPT. Now I am writing about using ChatGPT as a coding tutor.

Background

I have always struggled to find application software coding projects to improve with. I’ve not been driven primarily in that space, but as a result I have struggled to improve my coding chops. I do get to do some ‘proper coding’ in the day job but it is occasional and limited. Where I have tried to up my game with book and video courses I have found that the learning is never suitably paced. Typically it’s too slow at the beginning and then suddenly too fast - classic ‘How to draw an Owl’ Territory:

Read more →

Managing Permission Set Assignments by Organizational Unit in AWS Single-Sign-On / IAM identity Centre with Terraform

This article demonstrates how to use Terraform features new in February 2023 to comprehensively manage permission set assignments in AWS Single-Sign-On / IAM identity Centre across accounts and by Organizational Unit.

Note on Naming

‘AWS Single-Sign-On’ is a descriptive but fairly cumbersome name. In 2022 AWS decided to re-brand it to ‘IAM identity Center’, which IMO is confusing. For the sake of brevity in this article I am referring to both interchangeably as ‘AWS-SSO’

Read more →

Testing a Golang Application Written with Artificial Intelligence

Recently I wrote about writing a Golang application with a conversational Artificial Intelligence service - ChatGPT. This is a follow on article about trying to use a similar approach for testing. I am not presuming a familiarity with Golang for this article or claiming a high degree of expertise in it myself. The code discussed in both articles is here.

Context

I won’t repeat what I said previously about being a platform engineer, but one area where the discipline often differs from application software development is unit tests. Unit tests can be valuable in application development to check that behaviour matches expectations. Consider the following examples from wikipedia. We specify a feature, e.g.:

Read more →

Coding a Golang Application with Artificial Intelligence

A new conversational Artificial Intelligence service - ChatGPT - has been making waves this week. I tried using it for a small coding project and was impressed. I think it has some profound implications for the software industry.

(W) here’s the code

Of course it would be daft to not make the code discussed available so here it is with builds published for:

  • darwin-amd64
  • darwin-arm64
  • linux-amd64
  • windows-amd64

Background

I’m a Platform Engineer. I sometimes describe platform engineering in tech as like providing the foundations and utilities for a building- water, electricity, telecoms, etc. You don’t think about what goes into making those resources instantly available at any moment, you just expect them to be available when you need them and often only think about them when they go wrong. Just like these utilities however, your house will (typically) be uninhabitable without them, and without foundations it will not stand for long. Again, just like these utilities, once your organisation hits a certain size, you can’t just leave it to the public supplier, you need to involve yourself to ensure that you’re covered for your use requirement, if something breaks etc.- think of supplying power to a hospital or large factory for instance.

Read more →