Software longevity

Back
Mark McGranaghan (00:00:00)

Which by the way is something that's a little bit unique to digital systems versus classic analog systems. You know, if your ranch is rusty or it doesn't work as well, but it still basically works as a wrench. Whereas if you have one bit off and your software just crashes, you know, you're out of luck,

Adam Wiggins (00:00:15)

Hello and welcome to Metamuse uses as a tool for thought on iPad and Mac. But this podcast isn't about muse the product it's about Muse the company and the small team behind it. I'm Adam Wiggins here with my colleague Mark McGranaghan

Mark McGranaghan (00:00:35)

Hey Adam

Adam Wiggins (00:00:36)

Mark, You were giving us a very interesting little workshop at a team summit recently about the use of iPads in aviation.

Mark McGranaghan (00:00:44)

Yeah, so aviation is one of the most interesting and powerful use cases. I've come across for iPads in the wild. It's so powerful and important that folks are willing to spend 100, 200, $300 a year for high-end aviation related iPad software. So there's something right going on there, no pun intended. And as I've been exploring that world, there's a very interesting contrast and sort of technology sheer between these super shiny iPads and this new software that's being updated constantly. And the very old general aviation aircraft, you tend to see out there. This is the Cessna from the sixties, which by the way, is basically exactly the same as it was 60 or 70 years ago. And then they're being flown with these iPads from one to two years ago, and just very interesting to compare and contrast those worlds. And it led into this topic today, actually, because we were noticing that longevity of the aircraft versus the almost ephemerality of the iPads and the software and how much churn there seems to be in that world. So we wanted to dig into it on the show.

Adam Wiggins (00:01:52)

So a Cessna, which is kind of a small private plane is a extremely complex piece of technology. And also one that is used in very high stakes situation. If it fails, you fall out of the sky and die, and it has very complex controls as well, but those are all, I guess, analog is the right name, but you know, again, they look the same as they did in the sixties. Even new ones built today and the ones built in the sixties, they continue to essentially, you need to maintain them. You need to replace parts and upgrade them to comply with newer aviation regulations. But again, they haven't changed much in that underlying technology. And that's so wildly different from the world of not just iPad, but software and internet in general, where changes that an incredible pace. And in fact, that's probably desirable in this. What's the piece of the software. That's kind of the commonly used pilot software for

Mark McGranaghan (00:02:44)

Flight is the most common one. So

Adam Wiggins (00:02:46)

That's got maps, it's got weather, it's got flight routes, it's got locations of other planes. All of this stuff is being presumably downloaded or even streamed in through APIs. It's all very real timey and current information. And you want that in addition to just keeping up with all the new capabilities of the iPad. So you get, maybe that separation is nice. You get the benefit of the really fast-moving software and internet world. That's on this device, that's strapped to the pilot's knee, but it's completely decoupled from the safety, reliability oriented core instruments that are built into the plane.

Mark McGranaghan (00:03:24)

Yeah. Researching. This reminded me a lot of navigation in cars because my experience has always been if you buy, or if you see a car that has like built a navigation, it's always going to be bad because it was designed two to four years ago and it wasn't designed by a software company, but everyone just wants to use their iPhone right. To navigate. And they want to be able to plug it in and have it just their iPhone apps be displayed in the car. That's a good way of embracing the reality that there is some shear between those two layers in terms of how fast the technology tends to evolve.

Adam Wiggins (00:03:54)

So then our topic today is software longevity. And I think he can slice this two ways. One is the software itself and how long that lasts or how durable that is. And then you can also cut it the other way, which is, yeah, software's eating the world or is invading everything from toasters to cars. And how does software's dynamism impact the longevity of everything else as it creeps its way into the rest of our world. But as always, I like to start at the very beginning. So I guess first I have to ask what it means to you to talk about something being long lived or having longevity, whether it's software or a plane or something else.

Mark McGranaghan (00:04:35)

Yeah. I think it's the software being able to serve its stated purpose, which sounds very straightforward, but has several important constituent parts. First of all, you actually need the software. You know, sometimes we just lose it. It needs to run and needs to run correctly. It needs to have access to the relevant data needs to have access to the relevant pieces of the outside world in terms of API APIs. And it needs the appropriate substrate to run on an interactive.

Mark McGranaghan (00:04:59)

And there's probably other pieces that would comp with, but there's a lot of moving pieces that go into the software, actually serving its original and goal.

Adam Wiggins (00:05:07)

For me thinking about that word longevity, I tend to think about what is long, I guess, what is a span of time that counts as long. And of course it depends a lot on what you're talking about. So if you're talking about all of society, culture, humanity, then perhaps you're talking in the thousands of years, hundreds or thousands of years. So for example, the long now is a phrase that we used in the local first paper. We tend to use it internally on our team to refer to longer-term thinking, this comes from the long now foundation and the long now clock project, which is sort of this idea to build a clock that can keep time for, I think it's 10,000 years and it's basically an art project, but it's designed to inspire us to think longer term. And that can obviously connect to things about climate change or human culture.

Adam Wiggins (00:05:57)

And since humans are naturally inclined to think probably pretty short term, actually really short term, we think about the day that's ahead or the week that's ahead of us. Maybe at most the year that's ahead of us. We don't tend to think in tens or hundreds or thousands of years. So that's maybe a society level thing. I think for individuals and thinking about software as it impacts our lives as individuals, I think they're a human lifespan. That actually is a pretty good chunk of time to compare something to one interesting subreddit that I stumbled across years ago and still subscribe to is called buy it for life. And it's essentially people just posting photos or anecdotes of products they purchase. They're often something like a cast iron pan handed down by their grandfather or a pair of work gloves they bought 30 years ago. They're still working just as well as the day they were purchased.

Adam Wiggins (00:06:47)

And I think implicit in that is that there's some kind of inherent beauty or virtue in something that does have this long-lasting value versus something that's more flash in the pan. And it's interesting for me to try to tease apart the pragmatic aspect versus the yeah. That inherent beauty, which again, it feels to me like a virtue, but I'm trying to like dig a layer deeper and see if there's something practical that drives that maybe it isn't, maybe it just comes from a place that it seems right to me that something like products you purchase that their lifespan of the product could be measured in something that is a portion of a human's life. That's not thrown away in a month or a year even.

Mark McGranaghan (00:07:32)

Yeah. And I think we can come up with a few good reasons why longevity is valuable. The zero with reason would just be economic depending on how long lived a product is. It has different economics on the one extreme, you have consumables like toothpaste, you know, you use your toothpaste, it's gone forever. On the other extreme. You might have extremely durable things like stone tablets and masonry construction that could last hundreds of years. And in the middle you have the classic capital goods, durable goods, things like really nice hand tools or really well-maintained car that you expect to last at least several decades potentially longer. Like you were saying a lifetime or more. And the economics of those things are all very different. This goes a little bit back to the pricing podcasts that we had a while ago. Another thing is, I think just continuity. Like for example, if you run a business on a piece of software or you run your own creative process on a piece of software does real costs to the churn in that. And another thing would just be preserving history, you know, having access to the past. I think this is especially important with software because it's very easy to lose that both in the sense of the software itself and the data that it was manipulating.

Adam Wiggins (00:08:36)

And that side of it makes me think of some of these emergent field of sort of digital preservation archive.org is probably the, one of the biggest players. They're doing incredible work to save copies of websites, but also product manuals and old video games and indeed other kinds of software. And maybe that brings us to, when you do leave the world of durable goods, it brings us to the world of information and information longevity.

Mark McGranaghan (00:09:02)

Yeah. And one other point I wanted to add related information is I think there's a dimension of longevity, which is, and I'm going to make up a word here, roll over ability, you know, the ability to buy a new or get a new version of something that rolls over your previous state. So for example, like maybe you rebind a book or something in that way, roll over the pages, or you can put a new engine and transmission in a truck and you roll over the truck frame, but some stuff can't be rolled over classically that black box data to some opaque binary format. Now you're basically out like once that's off has gone. So I think that ability to roll over is an important dimension, even if it's separate from a single instance, having long longevity.

Adam Wiggins (00:09:42)

Yeah. Well information has this particular property that exists on a substrate or a medium of something physical Adams, but in the end, the information is not the physical thing. So the book rebinding example, you can either rebind the same book or in some cases.

Adam Wiggins (00:09:59)

Transcribe it completely. And as long as it's a correct and accurate copy, older books are an incredible artifact. Maybe they have margin notes, something about how the pages were made or whatever it does carry some history. There were the artifacts and objects of their own, right? But ultimately the contents, the words on the page are probably what we care about. And so on one hand information technology has actually been getting worse in the sense of the durability of the underlying thing, right? Stone tablets were very durable, but pirates and later paper were less. So then we go into the digital world and it's actually vastly less. So, you know, everything from CDRs to cloud storage, the failure rate is pretty high, but because of this, so easy, cheap to copy those bits from one place to another to replicate it, you potentially have the ability to roll it forward as far as it's worth your while to do so.

Adam Wiggins (00:10:49)

Right. Then if we sorta come to software as a special class of information and it is information, but it's highly dynamic. And I think that points to some of the particular challenges with it in, I guess that would be sort of the first category that I mentioned there at the beginning, which is the software itself and its ability to be long lived. And I think some of this is cultural. You know, the tech industry is dynamic. It thrives on change. That's sort of the nature of it and many things that are good about software and the internet and the tech world come from that willingness to embrace the new and almost an endless seeking of novelty. But the downside is, yeah, you get something that is not just an upgrade treadmill, but it just seems like it can be very much the case that the lifetime of a product or of data or of any particular piece of software can be measured in again, years and only a small number of years, which is comparatively just a very, very short time span. But I guess one question is why is that? Why should software sort of default beyond just sort of tech, maybe that's what it is, but I feel there's more to it than just culturally the tech world. Doesn't do the bio for life built to last thing, I feel like there's something inherent to the dynamic information, nature of software that makes it hard for it to be long lived.

Mark McGranaghan (00:12:11)

Yes, I think that's true. There's a lot of things that need to go right for the software to work. And you can see it. In fact, as something that's strictly harder than preserving simple textual information for the following reason, you have at least that problem to start. Cause you have the source code, then you need to preserve information about the dependencies. You potentially have the compiled source and then you have the whole runtime around it, very broadly defined, you know, the computer and the API APIs, maybe even the data. And it's just a very broad multi-dimensional. And in many cases, L specified state space that if you don't get your thing exactly right in that state space, it doesn't work at all. Which by the way, is something that's a little bit unique to digital systems versus classic analog systems. You know, if your wrench is rusty, you know, it doesn't work as well, but it's still basically worked as a wrench. Whereas if you have one bit off and your software just crashes, you know, you got a lot it's very high dimensional and it's very sensitive to errors in preserving the environment.

Adam Wiggins (00:13:14)

I think it's one wants to even think of a binary artifact. So here you've got a compiled executable. You can download it. You don't need to the dependencies, you don't need the compiler chain. And I think there's the feeling that should not just kind of work forever. But of course it says, you said it sits within this context of the system. If it's accessing files on your file system, it's calling out to API APIs on the network. It's accessing API APIs in the operating system. Even something like it just makes assumptions about what kind of hardware exists on the keyboard, a mouse as being standard input devices that exist. And then you go onto a touch device and those aren't there. And so over time, those fundamental assumptions and APIs and the system changes around it. And I think that's gotten dramatically more so with the internet where you might call out even something as simple as a static website that has a Google maps embed.

Adam Wiggins (00:14:08)

Well, now a big portion of that website, this big pain is depending on this exact integration to this service. And then that service is still online. And so I think that the one-off binary build shouldn't have worked forever does make some sense, for example, games, which kind of don't have as many hooks, let's say just like a single player game, not some complicated co-operative thing, but it tends to have very simple inputs and outputs displays things on the screen. Maybe it needs to save a high score file to the disc as opposed to productivity software, where all those little integration points, your drag and drop. And how do you open a thing in the browser and what all the different API is. They use to be a good citizen on the system and those change all the time. They should with an operating system that's growing and improving and evolving. But then that means the apps have to keep up with that. And if they don't, they fall out of date and eventually either become somewhat irrelevant or more commonly just stop working.

Mark McGranaghan (00:15:05)

Yeah. I think this idea of the complexity of the surface area and the API is very important. You mentioned games. I think we actually had it much easier back in the good old days of games where he's had keyboard input mouse input, you draw to a pixel buffer and that's it, the issue with games. Now, addition to other stuff around networking and multiplayer is the graphics pipeline is incredibly complex. And unless you end up emulating earlier versions of graphics pipelines, I think there's little chance that this stuff rolls forward. Whereas you could plausibly roll forward writing into a pixel bus offer. You can kind of go the other way. You can emulate a pixel buffer with our advanced graphics. API is today, but you have no chance of going the other direction. And I think this generalizes to API complexity broadly, and it's one of the reasons why it's so hard for software to persist

Adam Wiggins (00:15:54)

Games also make me think of the we're speaking briefly about the internet archive and yeah, you mentioned emulators and it turns out that games seem to be both something that people do want to preserve as a cultural history, but in a way, the best way to do that is not try to get the original hardware. But in fact, it's to have these emulators like Maine, for example, and it's really impressive yet you can play almost every arcade cabinet video game from the eighties, just in your browser with these kind of JavaScript, emulators that reload up the ROMs, which are basically the way they distributed binaries in those days and play them not quite as they were intended, because again, the hardware is different. Certainly the displays are different, but certainly we do preserve some that legacy. Now another one from games that maybe is your roll forward idea.

Adam Wiggins (00:16:45)

I really like what ed software does, which is they take their classics doom and quake and so on. And after about an hour, they're 20, 30 years old, something like that, they open source them. And that creates this interesting effect where maybe it's less, that people are playing the original, although I'm sure that happens, but then it becomes fun to just port it to weird places, right? Like you've seen quake that's been poured into like be a hundred percent ASCII. I think I saw a Twitter thread recently where someone managed to, they broke open. One of these digital pregnancy tests realized that it had a reasonably good processor and display in it and decided to like get doom running on it. Oh yeah,

Mark McGranaghan (00:17:24)

Yeah. It's the whole culture, you know, people running down the refrigerators and whatever.

Adam Wiggins (00:17:28)

Yeah. So in a way it has turned this over to remix culture and the internet. And so it gets rolled forward and become something that maybe people will experience none of this original form necessarily, but becomes woven into the cultural tapestry. Let's say through these fun little projects,

Mark McGranaghan (00:17:46)

I think this also speaks to the importance of that API environment that we're talking about either being specked or at least speckle for the software to be rolled forward or actively preserved either one, like we were saying old timey games, that's a pretty speckle environment. Like you need to build a compile C and relatively straightforward input and output system. And even if they games weren't designed specifically with a really clean interface there, you can basically back it out after the fact and then do the porting books and paintings, by the way, are examples of analog things that are very speakable. They mapped down to text streams and bitmaps respectfully, but some stuff like our modern software has much more difficulty being specked. And so it becomes harder to do that sort of rolling forward active preservation.

Adam Wiggins (00:18:34)

Now some have argued that perhaps part of the challenge here is the rapid pace of change of that underlying system of those API APIs. I think it's interesting here to contrast the apple versus Microsoft approach. So I think that, you know, Microsoft with Dawson later, windows really focused on backwards compatibility, quite an impressive level. I'm not exactly sure when the cutoffs are, but I think he could run dos stuff natively on windows till well into the windows lineage. And then much later versions of windows could run much earlier versions, but that's part of, I think, what also created the relative instability and unreliability of that line of operating systems, which is when you need to support every conceivable thing that's ever existed in the past and put it all together in the same box. Things get messy, pretty fast. And apple Elise with the iOS world has gone very much the other route, which is they basically are releasing new stuff, the new iOS update once a year. And if app developers don't keep up to date, you pretty quickly just fall out of the app store. So I experienced that with, I wrote a little puzzle game for the iPhone 10 or 12 years ago. I think it was on like iOS three. And yeah, my collaborator did some work to try to rebuild it with the new stuff and keep it in the app store. But I think bio has five or six. It just fallen out and it's shamed. There's.

Adam Wiggins (00:20:00)

No way to play this scan. That was a fun side project. And we spent some good effort on it's just sort of lost a history because you have this very kind of demanding operating system that requires developers to be doing active maintenance or else it tends to go away.

Mark McGranaghan (00:20:16)

Yeah. And while I don't doubt that there was some contribution in the windows situation to instability from needing to support all versions forever. I think what really gets you is when you need to do that and things that are really end user facing like visual in particular, because the strategies that you typically use for providing stability are not viable in that world. So if you look at things like Unix style server, operating systems, many programming languages, these are things that can have backwards compatibility for a long time, certainly decades, but the way they do that is they just never, ever, ever, ever break any APIs. And if you want to do something new, either a too bad or be call it thing too, and just keep thinking around forever

Adam Wiggins (00:21:00)

Python two versus Python three comes to mind. Oh

Mark McGranaghan (00:21:03)

Yeah. Well that one, that's a whole thing. There was an element of that of not wanting to break Python too, but even just individual methods. You know, when we write our own programs, we think of, oh, we need to change a method. Just like edit the source code. Well, not so much if you have the entire universe relying on thing one, who's got to keep thinking around forever and write a new thing to the people can call into. Anyways, you can't really do that with like a windowing system, for example, or a user interface paradigm, or how things look on the screen, right? You just got to pick something and do it. And the whole thing needs to go. And so I think it does become very hard to have that culture of very long lived durability that you see in some of these more systems he programs.

Adam Wiggins (00:21:43)

Yeah. Maybe that points to the fundamental trade-off there, which is durability in many cases means stability, which means less change, but changes how we get things that are new and better in the world of technology and computers and the internet is so at the very beginning, there's so much unexplored space and it would be a shame if we just said, okay, well, we've been working on making these computers here for a few decades now, pretty much how they are is probably, it is probably good for the next several hundred years. Let's just pull it there. That's not what we want to do. Uh, we want to keep changing and improving. So yeah, there just as a trade off between those two.

Mark McGranaghan (00:22:19)

Yeah. I think there's a very real tension. And I agree. We want to keep exploring. I do think that in adopting this attitude of exploring and taking risks, we are going to find some things that in retrospect were not good choices. And the intuition here is I think we're projecting these very dynamic system benefits over lives that we have historically associated with durable goods. You know, so we get the shiny new computer enabled software enabled thing, and it works great the first year. And we think things like this before we had computers, they worked for 20 or 30 years. Therefore this thing is gonna work for 20 or 30 years and it's gonna be awesome. And we've kind of projected that all forward. And I think in many cases, that's not going to be true. And so we're going to have to confront the downside of that. Trade-off

Adam Wiggins (00:23:04)

And do you think the downside there is mainly economic you buy the smart refrigerator seems great for the first year. Some cloud service goes offline. Suddenly your refrigerator doesn't work anymore and you gotta throw it out and get a new one. Or do you think there's something beyond the economic side?

Mark McGranaghan (00:23:18)

Unfortunately, I think there are several potential, very serious downsides. And just to give you a couple of one that we're already seeing is this issue of repairability or even modifiability again, if you look at these durable goods categories, like tractors is a very prominent example right now, traditionally people could service and maintain and change and modify and improve their own tractors. But as tractors increasingly become like iPhones, you know, there's these black box computers in many ways or highly computerized hardware, potentially you can't service them yourself. You can't repair them yourself. You can't modify them yourself. And that has all kinds of second order effects. There's also this whole thing about like surveillance, basically that I think we're sleeping on a little bit, but we don't need to go down that whole road today

Adam Wiggins (00:24:08)

On the economic side. I think there's also the incentive, which is basically apple and other phone manufacturers have done very, very well by always creating that shiny new thing that comes out of a year or two. And you think I want to upgrade. It's even often built into cell phone contracts and things like that that people want to get a new phone very frequently. And I think that's paired with sometimes the operating system upgrades or maybe the visual refreshes. It's almost more like fashion, right? The fashion industry found this very clever way to get around the fact that their IP isn't really protectable, which is they just make new fashion trends that totally changed for a couple of years. So then you like need to go out and get a new wardrobe. It's not that the clothes wore out necessarily it's that you want to be up to date and you look behind the times.

Adam Wiggins (00:24:53)

Stroll around in your bell-bottoms and those have been out of fashion for three years or whatever it is. Yeah. And so I think there's an element of this kind of a fashion desire to get the new thing that has driven a lot of revenue for these companies. I don't mean to imply that that's some kind of a cackling in the back layer, they're using this opportunity to prove and genuinely make their products better. You know, each version of the iPhone has that better camera, that better battery life, the bigger, brighter screen, it is genuinely better, but it's also the, company's a very incentivized for a high pace of change that has you buying new products all the time as opposed to something a longer-term. All right. So let's say for the sake of argument that we've decided that longevity in our technology products is desirable, whether we, because we think it's about preserving history or because we think there's inherent beauty in timelessness, or we think there's a good economic argument. What can we do as engineers and designers and product managers to make our software stand the test of time?

Mark McGranaghan (00:25:55)

Yeah. I do have some techniques there. I do have to preface it by saying that I think there is a large element of just draw the out here. I don't know if you've seen this before, but you got to sit down every week for 30 years and make sure the software doesn't break and that's sort of a necessary, but not sufficient condition, I think. Mm.

Adam Wiggins (00:26:14)

And the implication there is also just maintenance, right? And that maybe ties into some of our discussions about sort of software supported by subscriptions versus one-off payments, which is, it really is an ongoing effort and important ongoing maintenance effort rather than a one and done.

Mark McGranaghan (00:26:32)

Now that said, I think there are things you can do to make this maintenance effort much more feasible. One thing we've alluded to is this idea of narrow defined APIs because your software has to run in something and the broader, the more complex, the more ill specified that environment is the harder it's going to be to do this job of keeping the software running. And so this would include things like taking on few dependencies, minimizing weird binary compentencies that are hard to compile limiting the APIs that you access. Things like that. Another thing that I'm very big on is really focusing on data often, especially in this world of rolling forward over multiple generations. What you really want to keep is the data. This is more true, probably for like productivity type stuff than games, for example, but data is very important. And if you want to roll for data, you either need to adopt an existing open data format and existing data format, or you need to do your own, which can be done, but it's a lot of work. And I think that programs or program lineages, if you'd call it that embrace this idea of really focusing on the data so that, that can always be ejected and rolled forward into the next generation of software.

Adam Wiggins (00:27:47)

And one good example of that from my personal life is I've had the same calendar in some ship of thesis. Since I think I started using a product called 30 boxes, I don't know, 15 or 20 years ago, it was my first kind of like digital online calendar and then have migrated from one to the next is new stuff comes along and hopefully there are some standard formats here. I think it's caldav or something like that. There are also, I think these companies are often incentivized to make importers where you can kind of slurp down from the other company's API or whatever your existing calendar, but it would just make it much harder to move forward. I probably could, but it's just, I do put things on my calendar that are either annual recurring things, like, you know, pay the taxes, but also things that are really want to make sure I remember that are pretty far out. And I like having that history in the past. I like to know when a particular thing happened so I can look it up occasionally. So having that, my calendar is something that for me, feels distinct from the particular calendar software I happened to be using at the moment. Yeah.

Mark McGranaghan (00:28:53)

I also think there's a social slash community element here where if the thing that you're building is used and demanded by a lot of people, there are a lot of forces that are going to be pulling in your direction for that thing to be preserved. It's kind of like this data quality issue that I think we've talked about before in the podcast where the quality of data tends to be determined by how often and carefully it's read and how often and carefully it's written. And I think similarly with software, if you have many people constantly trying to run the program in different environments, that will encourage it to be come persistent and that, by the way, it goes down recursively to your dependency. So one of my favorite examples is escalate. If you use escalate to write even your semi-custom data format, if you write it into escalate database, you're absolutely going to be able to read that in 30 years, even if you do nothing yourself. Whereas if you roll your own format on desk, there's a very good chance that it will at some point become totally lost.

Adam Wiggins (00:29:49)

Yeah. SQL I think is a favorite example for us in many cases of sort of well-made simple software that does one thing and does it well.

Adam Wiggins (00:29:58)

They have a great page though, about their long-term support. I'll link that in the show notes where they list off some techniques like testing and making their database files cross-platform and disaster planning and not embracing hot new technologies with too much Gusto. But actually I think one of the things that points to that to me feels like maybe the most fundamental is to just think about longevity in the first place and just care about it in the first place. Part of the reason I thought this topic would be an interesting one. I think again, tech industry skews young, I, I think also have the same, what is it like gap in my thinking as a younger person, which was, I just didn't have the longer experience to be able to see what it is like for a software that I've created. That's in production for three years, five years, 10 years and longer.

Adam Wiggins (00:30:46)

And what happens over time? It's a part of what I like with the SQL Lite long-term support page is they basically lead with a statement of our goal is to support this until the year 2050. And so will they achieve that hard? No, but just by like making that statement, calling it out as a value for themselves and thinking about it actively, maybe this comes back to the long now clock idea, which is they may or may not succeed, but certainly one good way to not succeed is to not think about it in the first place. And so this makes it a first class concern.

Mark McGranaghan (00:31:20)

Yeah. I think that's a good point. You got to think about longevity for sure. And for me that circles back to this idea of data, my see people typically design and implement programs. Again, thinking for example, productivity type software, we tend to focus on the interface and the behavior because that's what users are demanding. That's what the short-term success of our business depends on. But the reality is that interface, including its implementation is going to almost certainly fully churn at least once, potentially several times over the course of software, but what's not going away is the data. And so one thing that I think is helpful for longevity of software is really focusing on the underlying data model. Not only in the sense of how it's stored, which is mostly what we've been talking about in the podcast so far in terms of text files or SQL Lite, but also what is the data model itself? Like if you were to draw out the SQL tables are equivalent, are the boxes and what have labels on them. It sounds really basic, but it's a step that I feel like people often skip over and then it leads to all kinds of longevity, headaches down the road.

Adam Wiggins (00:32:21)

Yeah, future-proofing I think is a word we sometimes use in general for talking about helping things be longer live, but certainly I think that's most notable in the data model. Precisely for the reason you said it's just much less changeable and harder to change and heavier weight to change

Mark McGranaghan (00:32:37)

Folks. You want a practical tip here. It is put a UID on everything, put a version on everything and only write new data. Don't plead all data. Yeah.

Adam Wiggins (00:32:47)

That is a deceptively simple list. But I think it probably reflects some pretty deep I'm guessing painful experiences in your career. Got any war stories for us?

Mark McGranaghan (00:32:58)

Well, the version one, I remember, well, cause you're the one who taught me. We were working on the Heroku runtime where the runtime builds applications into binaries and they are deployed and run and I was working on, but at the time was the unlabeled version, one of the system. And you point out mark, I'm sure at some point we're going to have a different version of how we package up these files. So I'm going to give you a tip, which is that you should write version one next to all of these. And then when we go to do version two, it'll be a lot easier and sure enough, that made a huge difference. And we did eventually get to version two and then 3, 4, 5, 6, I think after that.

Adam Wiggins (00:33:37)

And that was something at that point, you know, you were earlier in your career and I had had a few of those bumps and difficult data migrations and had realized that this one simple trick can save you some headaches. Now of course the future-proofing should be balanced against. What's usually called. You're not going to need it, which is over, I guess, engineering for something that an unknown future. So you're not trying to create a system that has totally flexible properties and tries to take into account every feature you might ever want to possibly add. I think that tends to create abstractions that just get in your way. It's more of the small, simple tricks that don't get in the way, but create opportunities to more easily make changes that you can't guess right now in the future,

Mark McGranaghan (00:34:23)

Right on the data modeling side. The advice that I always give is if you're just trying to accurately a model of the world as it really is because the world doesn't really is isn't going to change. It might expand as you add more features, right? But if you have incorrectly portrayed the world and your data model, and then you also go to model more of it, you know, now you have two problems, plus they're intersection three problems. Whereas if you had correctly modeled the part of the world that you were working with, it becomes relatively straightforward to extend the model to the new features

Adam Wiggins (00:34:53)

And to go just a little bit technical for a minute. The version thing is also making me think of, I think it's the image file format, maybe TIF, that's the one I'm thinking of, but this approach I think, has been used in a lot of different places. And I feel like you're doing a version of this in the local first sync infrastructure we're working on right now, which is to have chunks that have kind of a type in front of them. And so you can add new chunks to a stream of data or to a file that older versions of the software can load recognized. They don't know this chunk and they can kind of skip past it and just try to interpret the rest of it, the best that they can. You know, maybe this is something that web browsers do pretty well, which is you load a newer page in an older web browser. And if there's some unsupported features, it doesn't completely break. It just does its best to render it. Of course, the older the browser is, and the more new features you're using, the more and more sort of ugly and unusable the pages going to become, but it makes its best effort. It's not that it just gives up the moment. It sees something it doesn't understand.

Mark McGranaghan (00:35:52)

And this gets into the peculiar challenges of longevity with distributed systems software. We've been talking about longevity in terms of supporting the past with distributed systems. You've got to support the future because the future arrives unevenly. So that's the sort of challenge you deal with and you're working on distributed systems like the data synchronization layer from use. Yeah.

Adam Wiggins (00:36:15)

As usual Lincoln switch has some good research on the subject with the data lenses in the Cambridge project. And one of the things there is about sort of data migrations, but it's assuming you need to translate both ways you need to translate, I guess you say back in timed, older versions and forward in time to newer versions. But in fact, it's actually more complex than that because there may be just a branching tree of systems that all understand that the data in slightly different ways and the other small thing, it's sort of obvious in some ways, but I think it's kind of a Lindy effect of file formats. So the Lindy effect here, of course being sort of the idea that something has been around a long time probably will be around a long time. So I'm always a big fan of those flat file formats, PNG, JPEG PDF, plain taxed, because they've proven themselves to be durable over the long term and they don't always do everything you need, but where you can, it's really nice to just kind of go down to that simple common format. That's understood by many applications both now and in the past and potentially will be in the future, bringing all these ideas together in a very practical and a relevant area for us. How do we think about this for muse the product, the data aspect that you mentioned, but also the team and the company?

Mark McGranaghan (00:37:37)

Well, I think there are layers. I think that first layer in the sense of what's likely to be longest lived is from basically day one, we've supported flat file export. So you can export your board to a PDF or your images to a PNG. You can export your whole Corpus to a zip of a bunch of flat files of these types. And that way you'll know, you'll always at least have your data in this format that everyone knows how to store essentially forever. And yes, it's lower fidelity. Obviously you can easily edit it and like you can amuse, but that's sort of the trade off you make. There is you have access to this very durable format for all the data.

Adam Wiggins (00:38:18)

And I often find for archival purposes, I have put a lot of effort in the past to preserving things I've created, whether it's a essay or back when I used to make music, trying to preserve all those kinds of source files, video, similar thing. And usually I find the kind of flattened artifact read only is really kind of what I want in the long-term because I do go back to reference those things or look at them for inspiration or just for take a walk down memory lane, but very rarely want to edit even if I could. And so in that sense, just it's actually superior to flatten out to an immature video or

Mark McGranaghan (00:38:54)

Something. Yeah, that's a good point. In many ways, it's a benefit, not a downside. I think, especially when you have this habit of building up an archive, that's all on these handful of formats, everything is basically texts, PNG, PDF, maybe video MP3, you can browse it all together and you can do it very quickly basically and find, or the equivalent. Whereas you can imagine if for every picture you want to check out, you had to open up Adobe Photoshop or whatever you had to wait two minutes. And so it's nice just to have that very lightweight archival copy

Adam Wiggins (00:39:25)

Now for the apps kind of data itself. And we did toy early in the company with having it be part of our product to try to make that be something that's more like the native format that muse is saving too, is like a Dropbox folder, for example. And it just turns out that we couldn't achieve the things we wanted to either being on iPad or things we want to do around sync just wasn't compatible with that sadly. Right. But what we do try to do is really make the format that the app stores its data and on your device on iPad or now on the Mac, a format that is something we will.

Adam Wiggins (00:39:59)

Over the long term. And obviously we've only been at this what two and a half years, three years, if you count the research time, but already we've had people who have been around from the early days of the beta and we've had to, including our own personal corpuses, you know, I've got 20, some odd gigs of stuff and muse from my now years of doing all my thinking and strategizing in it. And the engineering team spends a lot of effort on carrying forward as we build major new things, whether it's something like the flexible canvas or whether it's something like, yeah, we, at one point converted the ink from raster to vector. We have another huge sort of data migration here with the introduction of sync that the team's working on right now. And it's a high stakes, but really important operation to bring that across. So ideally you could have been, and in fact, many people have been using these that whole time and you can go back and access your very earliest boards and everything is pretty much the same as you left it. You can still edit it. It's all still right there. Yeah.

Mark McGranaghan (00:40:58)

Now that data layer and rolling forward versions does become much trickier in a world where you have multiple writers. And especially if those writers are potentially not programs under our control. Certainly right now with music in the near future, the deal is going to be, you have Metamuse the app writing your data like a traditional cloud app would, but you could also imagine a world where, and we have in fact imagined a world where we have things like bots and end user scripting and plugins and extensions, and that world everyone's participating in the data model. So the stakes becomes much higher and you can't just roll forward the world when you deploy a new version. So that's why, as we were working on sync right now with an eye towards this world of scripting and extensions and so forth, we're thinking very carefully about the data model, because you basically need to support that forever or go through a very troublesome migration process. So we're trying to lay that foundation now, even though it'll be some time before all of those end results come to fruition,

Adam Wiggins (00:42:02)

Now that's the data. So how about the software, the app you download or the product more broadly over time? Is it important for that to be long lived? And how do we think about accomplishing that?

Mark McGranaghan (00:42:14)

Well, I think it's harder for that to be as long lived as the data, because that's less than our control. You know, you're participating in the apple ecosystem. As we talked about this ecosystem has certain characteristics that lend itself to being medium term life and practice, you know, apps, very individual builds of apps. Don't last a decade, even if the app itself does. So we do, we can, there, we have minimal external co-dependencies and we have no critical dependencies on third-party network services, except the ones that apple requires for any app to be able to be downloaded and paid for and so forth. But we are participating in that model of, you got to keep the app sort of up to date on that timescale of quarters or years as the underlying runtime environment, iOS and Mac changes.

Adam Wiggins (00:43:01)

Yeah, exactly. I would think of the software as in any particular build as being something that at least I hope would continue, you know, in some future where, and I know someone's running this in an emulator, for example, as long as the operating system API APIs are kind of what's expected. And again, we don't tend to use heavy amounts of those. We tend to keep it pretty light, but assuming we can read pencil data, for example, and the API that's expected that should still work. And a big part of that is the network side. So many apps on your phone, on your tablet and increasingly on the computer, they expect some kind of service to be online. And I tend to discover that because I really like working in offline environments. I like taking a long train ride or working on a plane or just going to one of our team summits in some weird rural location with weak internet.

Adam Wiggins (00:43:48)

And actually that's good. Let's see, be more kind of focused on what's in the moment, but then you really quickly get exposed to these weird timeouts and network errors and can't contact the whatever. And I'm thinking, why is this software even contacting the network? Those hooks are just so pervasive. Now we hardly think about it, but we're working from use to make it be very much. The network is optional. You get some nice benefits, but you don't need it once you're logged in. And then I would probably say, you know, that's any individual build of the software, but I would say on the product side, it comes down to more of a team and viable economic model and things that we.

Adam Wiggins (00:44:27)

Talked about extensively on the podcast here. But if we have a product vision, we think is good that the members of the team are invested in that term. And we've got a sustainable way to sort of fund it or work on that over the long-term. And it even ties in with something like just running the team in a sustainable pace, right? Avoiding this drive to burn bright, a burnout quickly that maybe does tend to come in the kind of hypergrowth world of startups. And instead be thinking a little bit more about, yeah, we want to be doing for this for awhile. We know great products take a long time to build, and that's not just like total personnel hours to build, but in fact, as wall clock time for people to use in the real world and get back feedback and expand and improve upon it. So running the team in such a way and having vision that we think that we're all committed to in the longer term, again, thinking in terms of human life scale spans, not months or just years, but maybe slightly longer than that. I think that's how we get a product. You can really depend on the longer term.

Mark McGranaghan (00:45:32)

Yeah. Another way you can think about this is lining up the reality of the software environment, what you're communicating to users with, how the team and the company is run. So it's not even necessarily that you want all stuff to be exactly the same forever. It's more like you want these promises to line up. So for example, as we're developing new features, you're really annealing them in. You're not saying as soon as you come up with an idea, oh, we're going to support this forever. Initially you do a beta and you say, we're going to support this for the beta. It's gonna be there for a few months and longer if it works well and not if it doesn't, but then when it kind of graduates to the next layer, it becomes something that you support longer-term that you have corresponding infrastructure around what the company to do.

Adam Wiggins (00:46:12)

Yeah. And from the perspective of customers, maybe some of that comes back to trusting the team and knowing the team, right. That's why I think it's important to communicate our values, our philosophies, and our motivations through writing through podcasting, where someone can dip into that world a little bit, get to know us and have some sense of what's driving us and where we're going and what we value. And if you share some of those values and you see a similar vision, then you can maybe trust that the software will change and evolve over time, but it will change and evolve in directions that are net will hopefully be an improvement for you. At least not a downside or a disruption. Whereas if you listen to us, talk for a few hours in this podcast and you think, Hmm, these folks are, you know, thinking about things in a different way than I think about things.

Adam Wiggins (00:47:03)

And then maybe it'll happen to be that the product does something useful for you right in this moment. But maybe over the longer term, it won't be evolving and directions that are as useful to you. So that's the reason in my own life, I try to use software as much as I can from teams. I know which it helps that, you know, I'm in the industry, I'm reasonably well connected now, thanks to Twitter and other sources. And then of course it's really to use your friend's software, you know, using the arc browser or something like that, for example, or some of the not boring apps from our friend. And he works as a couple of examples, but I also like that because yeah, it feels different when I know who's behind it and what they're trying to achieve. It feels good both to support their work, but also the sense that sort of equate like we're a team, but we're working together to reach a similar future where they're working hard to make this software that can serve my purposes and I'm giving them feedback and using the software and paying for it. And that's my contribution. And together, hopefully we work towards a better future.

Adam Wiggins (00:48:08)

Well, let's wrap it there. Thanks everyone for listening. If you have feedback, write us on Twitter at muse app HQ. We're on email. Hello at ... dot com. And of course we definitely appreciate it. If you leave a review on apple podcasts and mark, here's hoping that in addition to our business venture being long-lived that indeed this podcast will be long lived.