Thread by @JohnSumser

Maintenance. At the heart of #intelligenttools (#AI), deployment is a super-challenging question. Since all algorithms and models ‘drift’ (wear out), maintenance is a natural and important part of usage. (1/21)

Maintenance. But, maintenance is the hardest part of digital life. When our institutions train software engineers and data scientists, the emphasis is on the design of the new and problem-solving. Maintenance is just not as much fun as discovery. (2/21)

Maintenance. It’s not like all designers embrace maintenance. Maintenance is tedious, involves monitoring and preparation. Unlike the frontier spirit of current product development, maintenance plods, manages risk, restores, refurbishes, incrementally improves. (3/21)

Maintenance. What are good design practices? At the core, maintenance must be considered when design starts. Designs that do not include maintenance considerations are not sustainable. How will we fix, repair, improve, or replace this thing is a primary design question. (4/21)

Maintenance. Minimum Viable Products are minimally viable, in part, because they rarely include maintenance considerations. It’s getting to be too late when the MVP is released into the field. (5/21)

Maintenance. The actual quality of lasting code depends on the quality of the documentation. Can the team that follows the initial design crew make sense of their methods and decisions. (6/21)

Maintenance. The term ‘technical debt’ refers to the cumulative consequences of shortcuts taken in development. To meet deadlines or solve low priority problems lesser quality solutions get implemented. Not documenting the work is one of these shortcuts. (7/21)

Maintenance. The US Military has career fields and specifications devoted to getting maintenance right. They call the process of figuring out maintenance requirements ‘Integrated Logistics Support Analysis’ (MIL-STD-1388/1A) (8/21)

Maintenance. Side note. Did you know that I’m a Certified Professional Logistician (CPL)? It’s my engineering credential. CPL’s study and understand the tradeoffs involved in minimizing the actual life-cycle-cost of a thing. (9/21)

Maintenance. Maintenance is the largest overall cost of anything over the course of its useful life. Keeping an object, a system, a tool, a measure, or a software product in good working order is the object of maintenance. (10/21)

Maintenance. To get to the point, the maintenance of algorithms, models, heuristics, and code is missing from the ongoing conversations about #intelligenttools (#AI). It’s a really big deal. (11/21)

Maintenance. All digital things drift. They don’t exactly wear out like tires. Instead, they vary from their original purpose and intent over time. In enterprise SaaS solutions, the drift is often folded into the ongoing series of revisions made to the tool. (12/21)

Maintenance. ‘Drift’ is easier to swallow than ‘wears out’ or requires maintenance. But, it means the same thing, more or less. (13/21)

Maintenance. All of the current #intelligenttools (#ai) ‘learn’. The data changes the tool. This is the heart of drift in the kinds of AI deployed today. The tools need maintenance (14/21)

Maintenance. But, there’s a problem. It’s very difficult to determine whether an intelligent tool has drifted and if so, how much and then, whether it matters. #intelligenttools (#AI) don’t have the equivalent of engine warning lights. They should. (15/21)

Maintenance. Tools require calibration and recalibration. In knives, it’s called sharpening. In cars, it’s alignment. All #intelligenttools (#AI) require calibration. (16/21)

Maintenance. If it’s technically possible (and there is some debate about that), calibration should be a part of every intelligent tool. You should be able to understand its behavior in a quantitative and qualitative way so that you can tell if it needs maintenance. (17/21)

Maintenance. But that’s not yet the state of the art. So, what should you do instead? For starters, always ask to see the code for models, algorithms, and the applications they call home. Have a software engineer make a determination about the embedded documentation. (18/21)

Maintenance. Always ask the vendor to explain their approach to maintenance. In your heart, you’ll be hoping that they designed the thing for maintenance from the beginning. You probably won’t hear that. (19/21)

Maintenance. Instead, get a feel for how they plan to maintain it. Ask yourself if you could do that maintenance should they fail, go out of business, or stop supporting the product. (20/21)

Maintenance is where the actual cost of the tools you buy shows up. 21/21

Unroll @threadreaderapp Thanks.