In this episode, we talk to Alan and Micha about Javelin and Hoplon.
The show is available on iTunes! You can also subscribe to the podcast using our podcast feed.
You can send feedback about the show to podcast@cognitect.com, or leave a comment here on the blog. Thanks for listening!
EPISODE COVER ART
AUDIO PRODUCTION
PRODUCER
Our theme music for this episode is Thumbs Up (for Rock N' Roll) by killthenoise with Feed Me which was used under a Creative Commons License.
In this episode, we talk to Alan and Micha about Javelin and Hoplon.
CRAIG: Hello, and welcome to Episode 112 of The Cognicast, a podcast by Cognitect, Inc. about software and the people who create it. I'm your host, Craig Andera.
Well, we have a bunch of Cognitects that will be out and about in the world. I want to mention a few of the places they'll be this month. This is all in 2016, November of 2016 specifically.
First up, there's a meet-up: Clojure PDX in Portland, Oregon. Stu Halloway will be there. Rather, he'll be giving a remote presentation on agility and robustness in clojure.spec. That's November 3rd. The venue for that is Portland. Although, like I said, it's a remote presentation.
Mike Nygard will be doing a bunch of stuff in November. He'll be all over the place, actually, doing a lot of cool talks. The first stop for him is the DevOps Enterprise Summit, which is in San Francisco. That's happening November 7th through the 10th at the Hilton San Francisco Union Square. He will be presenting at that conference.
He will be doing training at the O'Reilly Software Architecture Conference also in San Francisco. That's happening November 13th and 14th. The title there is Architecture without an End State. It's a two-day course, again taught by Mike Nygard. This, by the way, is not Clojure specific. You can search on that for more info.
We've got a lot of things to talk about today, so I'll leave you to discover those on the Internet.
More Mike Nygard in Iceland at G/O Digital. This is November 16th at the Hilton Reykjavik Nordica. He will be speaking there at that conference.
Carin Meier will be speaking at Ohio DevFest, which this is happening Saturday, November 19th at the Tangeman University Center.
Of course, the Clojure/conj is coming up. That is, I suppose, technically in December, although the training course is happening immediately before. We have a Datomic course. I'll be teaching the Clojure course. Those are at the very end of November. The conference itself is December 1st, 2nd, and 3rd. This is happening in Austin, Texas. Still tickets available. You should go to Clojure-conj for information about the speaker lineup, which has been posted, and to buy tickets. Sponsorship opportunities are still available, so lots of good stuff at Clojure-conj.org.
Finally, I want to mention InClojure. This is India's first ever Clojure conference - very exciting. This is November 26th at the Hotel Novotel in Pune, India. We've actually had a whole episode where we talked about our producer Kim Foster's visit to India and about the Clojure scene there. Obviously that's coming along very nicely since they are now having a conference. You can check that out at InClojure.org. Definitely check that out. That looks to be exciting. We're certainly psyched that that's happening there.
But that's a lot of stuff, so we will leave it with that and go on to Episode 112 of The Cognicast.
[Music: "Thumbs Up (for Rock N' Roll)" by Kill the Noise and Feed Me]
CRAIG: Okay. Are you ready?
ALAN: Yep. Ready.
CRAIG: Yeah, you were ready all along. I was the one that clearly wasn't ready. So here we go.
Welcome, everybody. Today is Tuesday, October 18th in the year 2016, and this is The Cognicast. We are very, very pleased to welcome back, right back in fact, as it turns out. We just shipped their episode, their previous episode today. We're recording again right after the release of that. I'm talking, of course, about Micha Niskin and Alan Dipert. Welcome to the show, Micha and Alan.
ALAN: Hey. Thank you, Craig.
MICHA: Good to be back.
CRAIG: It's really great that you were able to record again. I mean it's not that soon for us. We recorded that episode way back a couple months ago, but it's perfect timing, in my opinion, to have you back on. You know we just got done talking about – our audience will have just, in the last couple weeks, have heard you talking about boot. They're well aware then that that conversation – we left off before we could talk about javelin and hoplon, which are two other extremely interesting technologies that I have been using in my free time projects quite extensively over the last few months.
Of course, we will cover other things as they come up. The two of you are both just endless fonts of interesting information, anecdotes, high jinks, so forth and so on. So I'm sure we will have another great conversation. I'm looking forward to it.
But, of course, before we get into that, we always ask at the beginning of the show a question about art. Specifically, we ask for our guests to share with us some experience of art, whatever they think that means. Since we had Micha go last time, which resulted in, I think it's safe to say, the creepiest cover we've ever had on any of our episodes ever – it was also awesome, by the way–
MICHA: Bam!
CRAIG: Yeah. We had you go last time. That resulted in that. This time we're going to ask Alan to answer the art question. Alan, would you like to share with us an experience of art?
ALAN: Sure. Maybe it's a kind of performance art. I mean everything is art. Nothing is art. But I would call this art.
As you know, I'm kind of fascinated with space travel and space exploration and, you know, the heyday of space travel, the Apollo program, stuff like that. I've always been interested in that and, when I was a kid, I wanted to be an astronaut and a pilot, but I think, in my adult life, I was sort of reinitiated with Russ Olsen's incredible talk about sort of the developer engineer legacy and the relationship of that to the space race. I'm sure you know what I'm talking about.
CRAIG: Oh, yeah. That talk is excellent. People should go look up Russ Olsen's To the Moon to see exactly what you're talking about.
ALAN: Yeah. Amazing. I was really fortunate to have seen it live, and it was just really moving to hear the story told from someone who was there, who was there as a child, you know, a particularly imaginative, engaged child talking about what it was like to be in the same world while that was going on. As someone who existed in the future, I've always had this sense of nostalgia about it, so it's like ten times more awesome probably than it ever was.
But obviously you know the space program is not really a work of art. But I think, like any big endeavor, there are works of art performed routinely. One of the neater things that I've seen related to the space program that was kind of an artistic interaction was – and this is kind of – the astronauts actually have some artistic flexibility, especially in the '60s because the astronauts would be on TV, and they would be on the radio. NASA, amazingly, did not really script what they were going to say. They gave the astronauts a fair amount of leeway into what they were going to say on these shows, which is pretty incredible, I think. The first humans doing this basically are just given an open mic and asked, "How do you feel about this?" because everybody really wanted to know, like, what's it like to be further away from earth than any human ever has been. And what would a person say to that?
I think, by and large, the astronauts were performers and artists because they were probably the only people in the world who were capable of both doing that mission and also having, like, that casual tongue in cheek kind of test pilot sort of worldview. So they could be like, you know, flying into the sun to their doom and crack some joke, you know.
CRAIG: Is it warm in here?
ALAN: Right.
CRAIG: Yeah.
ALAN: Is that just me? Did someone turn it up, or are we going to die in ten seconds? You know?
CRAIG: Yeah.
ALAN: And I'd always admired that kind of humor and that kind of affect. If you've read The Right Stuff by Wolfe, he talks about that a lot and the whole Chuck Yeager personality meme. That's kind of the context.
The piece of art in particular that I'm thinking of, there was a mission Apollo 8, which was in December of 1968. We landed on the moon in, I think, May of 1969.
CRAIG: July, I believe.
ALAN: July? That's right. Apollo 8 was one of the precursor missions. It was the first mission to orbit the moon. They did not land on the moon, but they went all the way out there. They orbited it, went around the dark side, and came back.
As I understand it just from history books and stuff, 1968 was a tough year for everybody with Tet Offensive and RFK assassination and Martin Luther King was assassinated in that year too, so everyone was on edge - a very crappy year. These astronauts on this Apollo 8 mission were kind of in a weird spot because they were supposed to say something nice, and it happened that they were headed to the moon on Christmas Eve of 1968, so everyone was tuned in anticipating what they were going to see and, I imagine, hoping to be distracted.
There was an interview, later I saw, where they explained why they made the selection. But basically the astronauts chose to read from the Book of Genesis, like from the beginning. And there's this very moving piece of video where they had the spacecraft camera aimed at earth with an odometer that was rising: 172,000 miles, 173,000 miles. You can tell they're going at super fast speeds. The guy is just reading from the beginning of the Book of Genesis. In the beginning there was nothing and then God moved over the face of the waters and so on.
It's kind of – I don't know. It's music video like in that it seems almost too beautiful to have happened for real. I don't know. It was an artistic experience, and I definitely would count the astronauts who had that idea as artists because I would have no idea what to say. They said, hey, you're going to the moon. What do you tell the folks back at home?
Yeah, I don't think art is necessarily an artifact that you make because art, I think it's more like when you look at something, you might consider it to be art because of what it does. So a cool part of it is there is kind of an artifact because there is somebody who recently made a music video that mashes up a kind of spacey, upbeat tune with the NASA footage of the dude saying this. The artist is Lindstrom. He's like a Norwegian – Norwegian space disco is the genre.
But there's this kind of weird, spacey, futuristic music playing and then the astronaut reading from Genesis, and it's pretty surreal. If you search for, I think, Lindstrom space Apollo 8 on YouTube, you'll hit it. It's a very crazy, interesting video.
CRAIG: Very cool. We will, of course, put a link to that in the show notes so people can find it easily that way too. Well, it's interesting to me. I mean, so I agree with your statement at the beginning there, Alan. It's like, well, what is art? Art is everything. Art is nothing.
But the definition I always kind of reach for just to have something to hang onto is something whose intent is or effect perhaps is to make you feel something, to make the observer or participant feel something. The space program is clearly, clearly, like, a very strong intersection of emotion and technology for many people. I mean it's the pinnacle of technology in many ways, and yet people also have this really strong kind of emotional attachment too. I think your example is a great, you know, almost the pinnacle of that, right? Like how could you – that's clearly something that is definitely about the feel of something, the emotional impact of a moment combined with technology.
ALAN: Yeah.
CRAIG: Admittedly, technology that is less powerful than what we all have in our pockets right now, but more impressive nonetheless.
MICHA: Yeah. Yeah, no, I think by that definition, you know, something meant to impart a feeling.
ALAN: I don't necessarily think the intent is at all relevant. I think it's the effect that makes it art, not the intent.
CRAIG: I'll buy that.
MICHA: Yeah. I mean there's multiple definitions. I mean some things, some things you can call art retroactively.
CRAIG: Yeah, sure.
ALAN: I think that, yeah, for me anyway, I think that's the necessary and sufficient condition. I don't think you can actually make art. I think it can be art, you know, if a person experiences it in a certain way. You know?
MICHA: You're saying it's like eye of beholder kind of thing.
ALAN: Yeah, just like, you know – yeah. I mean I think we have math and science and whatnot to explain the world, but we have art to explain ourselves. That has to be, I think, an internal thing that the person experiences that. You can't really describe it in terms of logical systems.
CRAIG: The distinction that comes to my mind when you say that is – so this is what I talk to my kids about is I think there's a confusion a lot of times. I'm not saying that this is something that either of you is doing, but just as an interesting phenomenon between kind of an artifact and an event. Like I said, well, let's say that we all sat down and played a game of Monopoly. Then we put the game in the box, and we put the game on the shelf.
I would say, "Well, where is the game?"
They would say, "Well, it's on the shelf."
And I said, "No, no. I mean the game we just played." Right?
Like we all sat around and we can talk about that. We played at this time. We played at that time. Where is that? Of course, the answer doesn't make any sense, but I think that there are times. That's kind of the point of that exercise that I ran through with them is to be able to differentiate between artifacts and events because sometimes we talk about things like where is this thing, but really it was an event. It was a process, and it doesn't make sense for it to be in a place. Like where does the light go when you turn it off? That type of question.
Anyway, sorry. No, that's a really cool story, Alan, I mean a really cool – I'm glad you related that. That's very good and it's good observations from the both of you on that.
I love this part of the show, but we do, of course, have other things that we would like to talk about as well.
ALAN: No, no, no. Let's keep going with this.
CRAIG: We could. I know we definitely could.
ALAN: No, no. Let's please not.
CRAIG: The only thing that stops me from doing that is that we were in the middle of what I thought was a really interesting conversation last time.
ALAN: Yes.
CRAIG: We didn't get a chance to finish it, and so as much as this is interesting, I'd love to loop back to that.
ALAN: No, totally. I'm just kidding. Yeah.
CRAIG: No, but right. I get that. Alan, you're a funny guy. Funny guy, Alan - real funny guy.
ALAN: I'll shut up now.
CRAIG: No, no, don't shut up. That's kind of the point is for you not to shut up. If you shut up, the show gets really–
ALAN: I'll shut up and then Micha can talk.
CRAIG: Well, that'd be okay. That'd be okay. Anyway–
MICHA: Alan, can you do anything right?
CRAIG: The last time on the show we basically covered the first leg of the tripod of technologies that I kind of had you on to talk about, which the three are boot, hoplon, and javelin. I don't know if that's the right order, but we talked about boot first, which makes sense to me. It's kind of, in my opinion, the right place to start. But we didn't get a chance, because there was so much interesting stuff to go through, to talk about javelin and hoplon.
Now, if I was going to guess, I would imagine that it makes sense to sort of jump over to javelin first, but maybe you, as the creators of these technologies, have a different opinion. I'll leave it to the two of you to decide how we should approach that. What we would like to do, I think, is maybe explain what it is for anybody that hasn't had a chance to encounter it or not. But then I have a bunch of questions I want to ask you about it since I've been using it a bunch.
So all of which is a long way to–
ALAN: So, yeah.
CRAIG: Go ahead. Go ahead. Go ahead.
ALAN: Yeah.
CRAIG: Yeah.
ALAN: No, I have a thought about how to lead into this.
CRAIG: Okay.
ALAN: The last time, if I remember correctly, we talked a little bit about the origin story to, like, set up the background and the kind of things we were facing that resulted in the thing.
CRAIG: Mm-hmm.
ALAN: I feel like I told most of that story, but I feel like in this case Micha kind of owns the origin story because he's the one who did the original work on this technology at the Fresh Diet. I think he probably went through the most evolutions of it. I'm talking about the white labeling problem and stuff.
MICHA: Yeah, yeah, yeah.
CRAIG: Awesome.
MICHA: Sure.
ALAN: Maybe Micha could start there and then I could.
CRAIG: Yeah. I love origin stories. Go for it.
ALAN: All right, cool.
CRAIG: Yeah.
MICHA: Yeah, so originally I was working with Ray Willig at the Fresh Diet. He also, like, we started using Clojure there, and we had really, really good – a good experience with Clojure. This was in, like, maybe 2012, something like that.
The specific problem that we had, the company delivered fresh, gourmet meals all around the country, so we had kitchens all over the place in all these different cities, and we did the entire stack from sourcing fresh ingredients. We had our own deliveries. We did routing, everything, and we had, like a very full featured interface that the customers could use to figure out which. We would publish menus that had lots of choices, and they could choose different things that they wanted. They could also adjust them saying they didn't like things or, if they did like things, they could have extra, like extra peppers, whatever.
Also, there were a ton of more and more complex promotional, like marketing promotions. They would involve things like you can order a plan for a month, so it's a month's worth of food. But you don't have to consume it all. You could only get, like, a day here and then, like, next week you get two days of food and so on. So you could order 30 days worth of food and then, if you want to quit within the first week, we prorate it according to this formula, and then if you want to quit with the second week.
Anyway, they kept making more and more complex marketing type models that we would have to implement. Also, they were reaching the size where they had to start modularizing the business because they already were doing a lot of direct business with customers, but there was a lot of money also in being the backend to other people, so like celebrity chefs, for example, would want to make their own front end and use our services because we had the domain knowledge to actually make the food and deliver it, which is pretty complicated. Anyway, so our system was this, like, 250,000 line PHP application that basically mediated every single aspect of operations of the company.
They're like, "Hey, Micha. Can we, like, just white label this stuff?" There's no way. You cannot. It was all, like, HTML being admitted by PHP with, like, hundreds of CSS things coming in and out, so it was just not possible.
We had to think about how to make a system that would be flexible enough to allow different customers to have different workflows. An example of a specific case of this was we had a customer who wanted to use us with a nutrition focus rather than the focus that the Fresh Diet had. That means that the user interface would be kind of centered around nutritional, like this nutrition database view of your meals, which was unlike what the Fresh Diet itself was doing.
And so how do you start to reconcile the two? It's not just changing CSS or making colors different or whatever. It's an actual workflow change. And so we wanted to think of a way to make basically a framework for making applications on top of our service. It had to be done such that we could make a new white label site within a certain amount of time in order to get the deal, you know?
CRAIG: Mm-hmm.
MICHA: We looked around at all the different things that were available and tried experiments. We did a lot of weird things too like using Jekyll to generate a static site, which then used Knockout to talk to the backend - weird stuff like that. We ended up realizing that we needed to separate completely the workflow part, which is basically the state of the application and the application as a state machine from the user interface, the presentation, or what you see on the screen because a lot of the aspects of the workflow are business logic, business concerns. Those are constant. Those will be in any implementation, but they have to be in the client too. In other words, a person needs to select whether they're vegetarian or not before they can look at their menu, for instance, or something like that. These aspects of the workflow are universal, and they always need to be maintained by the system. We ended up eventually using ClojureScript, and hoplon basically came out of that.
ALAN: And I think the influence of the spreadsheet model, too, it seemed like.
MICHA: Right.
ALAN: Because, yeah, I think one of the things that – so when Micha did most of this thinking and initial implementation, I did not work at The Fresh Diet, but I was communicating with him on the side channel. It seemed like one of the things that he arrived at was the idea that you needed to separate the workflow, in other words the business. You have a business, and there's domain knowledge with the business. There are certain operations that are valid or invalid given some configuration of domain data. That's a separate thing from what the person can see and how they're allowed to influence that workflow.
One of the things that Micha ran into that he very excitedly told me about, if I remember right, was that that's basically what a spreadsheet is. If you consider a spreadsheet and charts based on a spreadsheet as a workflow and views into that, like mediated views into the underlying workflow. What you end up with, and I think an example he said in a previous talk–I remember Micha talking about–is like if you're at a business and someone comes up with a spreadsheet. Let's say someone in the accounting department comes up with a spreadsheet, and the spreadsheet has the financials for the past three quarters.
But then someone in the marketing department wants to make charts for their presentation. They don't understand the accounting bit, but they can look at the existing spreadsheet and hook up new, different kinds of charts. I guess the modern term for this is mash-ups. Like you mash up the data you got from the accounting department with data you get from the business analysis team or whatever, and you come up with a derivative diagram. What you end up with is a directed acyclic graph of data and dependencies and visualizations. The visualizations are the leaves. Those are the things that the end user sees. But underneath is a DAG of views into the state of the workflow.
CRAIG: Mm-hmm.
ALAN: Then the part that spreadsheets don't have as much, I mean they do have this if you are willing to tango with PB scripts, but an input thing other than manipulating the cells directly. Adding buttons and behaviors to initiate workflows. That's something that we take on with hoplon and javelin too.
MICHA: Yeah. I think there's an interesting division between. There's probably an official word for this, but I think of it as resident programs and applications that perform work. The backend of our application, for example, the thing that runs on the server, it's the request response type of model. It's not a resident program that's containing a bunch of state that can be inspected individually.
A user interface, though, or an editor or something like that I think of as like a resident system in that there's all this state that's in there and the user might be looking at a lot of it at once. With a request response like a database, say, you get all these requests coming in and you want everything to be lazy rather than eager. In other words, I'm not going to – like in a database you have views, and views are not really tables, usually. Usually there's some kind of like compiled query because you don't want to do the work of updating every view when a new row comes into the database because that would be very expensive. Whereas with, like, a user interface or a resident type of program, you actually do want to do that work because you don't know what the user is looking at.
ALAN: Right. I think maybe the difference between them is the entry points or the things that you choose to make visible or not. Like when you have a service that multiple users are going to be using, like your multi-tenant or multi-client, it's important that you don't expose all of the data to any one peer or client as opposed to in the JVM or Emacs where you have no idea which pieces of data that the program has accumulated that they're going to want to look at or not. You make no effort to protect or hide data from people, at least in sane systems.
MICHA: Yeah. I mean I think for the two types of applications that you have, you have two different types of sort of stateflow models. For a user interface where a lot of state needs to be accessible, you have a situation where you have a database that's updating the views as soon as new information comes in. Meaning it's pushing it out.
For example, well, I guess we could look at the other side now. The other side is sort of on the backend like a database, say. You don't actually want to do that. If you have a million users and each user can see their own stuff in the database, every time a new piece of information comes in you don't want to update every view for every user. Even if a piece of data might affect a bunch of users, you wait until they make a query. I think that's kind of the separation.
If you have a situation where you want to eagerly push out changes everywhere, which is, I think, what you want to do in a UI, then you could use something like javelin. Whereas if you're on the other end where you want to resolve those things lazily, that's something, a place where core.async would be good because you're feeding data down a pipe, so that's where you want queues and things like that.
But on the other end, in the UI, you don't really want a queue. The thing needs to fan out and constraints need to be satisfied. You have a bunch of constraints, so I think that's basically how we ended up with javelin was the need to not use queues in the front end because what we really needed was the spreadsheet.
CRAIG: Let's dig down that a little bit because there's a couple things in there that I'd like to unpack. What is it about queues that's inherently bad? We do asynchronous things in the front end all the time, even in a reactive metaphor like javelin. Certainly when we're talking off the box, right, that's inherently asynchronous.
But even when I've written UIs, and I'm far from an expert, so I'm more than willing to believe that I'm doing something wrong, I do in fact employ queues. I mean Web workers, I use those in my UI to get certain properties. Maybe that's not quite what you're talking about, though.
MICHA: No.
CRAIG: I'm curious, though.
MICHA: I don't mean to say that queues are bad because, of course, a queue is like one of the fundamental building blocks that we use to build applications. But I think there are times when you need to push out changes and there's times when you want to pull changes. I guess that's not – I'll give an example. With a spreadsheet, for example, when you change a cell, it pushes updates to all the other cells, which are satisfying constraints. All the formulas are kind of like a constraint solver. It pushes out, all the formulas update, the state sort of atomically changes from one state to the next, and all of your charts update themselves.
CRAIG: Mm-hmm.
MICHA: What that eliminates is the type dependency. You don't care necessarily – like all the cells know when they need to update. In fact, at the end of the day, it's as if they had all updated atomically in one instant because no cell can see any other cell other than in the latest state that it was in. For example, if I have cells A and B that are input sells, and then I have some cell C, which contains the sum of A and B, C will never see A in an older state than B, say.
CRAIG: Right, right.
MICHA: I guess if I update – yeah, so each cell only sees the things that it depends on, the cells it depends on, after they've already updated, if they're going to update.
CRAIG: It's really about atomicity and, of course, if you have a queue, you can't guarantee that, right? You've gone from–
MICHA: Right. If you have three queues, how long do you wait until all the results are in before you act on it, say?
CRAIG: Right. Right, right, right.
MICHA: Right?
CRAIG: Okay. Okay.
ALAN: Yeah, I have experienced this. Well, before Micha started working on javelin and caught the spreadsheet bug, I was actually at Relevance where I was working on a system that was an online video conferencing system. The metaphor we picked for modeling state transitions in the UI, which I think it was CoffeeScript in a browser at the time, was the EventBus, which is kind of the classical pattern for modeling events in a UI. I think that's when I sort of hit on this problem of suddenly time becomes important again in consuming – like a queue consumer is kind of like an infinite loop in the sense that it's a process that has its own lifecycle because it's consuming from this queue. Suddenly it becomes important, you know, how close the messages are to each other, like flash messages for example, like places in this UI would put a message on a queue if they wanted to show the error connecting dialog.
Now obviously we don't want to spam the user to fill up their screen with these error connecting dialogs. If one exists already, then we shouldn't show a new one. What that means is now we have a little accumulator in there, a stateful accumulator that says, "If there's a thing showing when I receive a message, re-enqueue this message to show up in 30 seconds," or whatever, which was less than ideal. I mean granted I may have been just doing it wrong at the time, but it was definitely evident to me that there was this temporal, like once I admitted queues or, in this case, a bus, which I think they have the same properties as far as UIs go in that there was no implicit ordering of events, you need to bring the ordering yourself.
You needed to do that by accumulating things in places and then have time out policies and stuff like that. Instead of being a nice graph of dependencies, the state associated with the way the UI looks becomes a graph of accumulators and policies and retry policies. It was ungainly.
CRAIG: Okay.
MICHA: A lot of the real strengths of queues is when you need to farm out work to independent, separate processes that don't necessarily know about each other. You have workers, like a Web worker or whatever. Things like that are great when you have some process that can be done by any anonymous worker, and you just need to make sure it gets done. That's what's really great about queues. Then you can get back pressure, and you can manage all these other little threads or processes or whatever.
But with user interfaces, that's not the case a lot of times. A lot of times, like the place that's showing you this flash message or whatever is a particular place. It's not like any element can just pick up this job of showing an error. It's like it has to be in that particular place.
ALAN: Right. It's not necessarily. This is, I think, inherent in the problem of the UI is that the UI is a singleton thing. The human being can only perceive so much stuff, and that stuff constitutes one single place per the application. The place in the UI where the flash message shows, for the purposes of the user's visual field, that is a singleton place.
MICHA: Yep.
ALAN: It is a single place in the person's field of view where a thing may or may not appear. That's just not appropriate. Queues aren't appropriate there because they're applicable when, like Micha said, you have work that needs to get done, but you don't care when it's done. With UIs, it's more important when it's done.
CRAIG: True.
MICHA: Yeah, and also the progression of state needs to be – I think you need to think a lot about – that's mostly what I think about when I make a UI is like how to state progress. And so it can progress either from – well, so the user interface, at any given time when you load the page, it'll be quiescent. It will have loaded everything that it needs to initialize and then it waits for the user to interact with it, usually, unless there's some kind of connection with the backend that might do things. The UI is at rest. The only thing that will transition it to a new state is something from the outside world, from the environment, so either the user interacts with it or the backend or some interaction with, you know, your database or whatever.
I think, when you have a system like that also, and all the elements that are on the screen, all the widgets, they're stateful as far as the user is concerned. Those states are satisfying some constraint, meaning they have some relationship to the state that's in the front end. Now the state might change, so in other words the user might do something and some interaction might happen with the backend. Maybe the two of those, together, interact with each other. Meaning the user says they want to create – they want to buy a shoe, and maybe the backend says there's no shoes available or something like that. But that's when it becomes really useful to have formulas that have the dependency graph because that can allow you to not do extra work. I guess the guarantees of javelin, we're kind of getting into what the guarantees are.
CRAIG: That's cool. Yeah.
ALAN: Well, and I guess javelin, too, it's a ClojureScript library that you use from ClojureScript, and it gives you this spreadsheet like computing environment for building and arranging these state machines/spreadsheet-like constructs that are aligned with this philosophy that we're unveiling.
MICHA: Yeah.
ALAN: Yeah.
MICHA: We should go over the–
CRAIG: Please.
MICHA: –what are the constraints or the features of a spreadsheet.
CRAIG: Yeah, yeah. Please, go ahead.
ALAN: Yeah, so your average spreadsheet, it's important to make a distinction between the spreadsheet that you have in mind right now, which is this grid of cells, and what we say when we talk about spreadsheet, which is like the underlying computation model. A typical spreadsheet like Excel, it has a field, X and Y dimensions, and that's your namespace, more or less. That's how you enter values and see values.
Then there's the computing model underneath that, which is the relationship between these cells and the constraints that mediate the relationship between the cells.
CRAIG: Right, which isn't rectangular even a little bit.
ALAN: Right. Right. Yeah, it's sort of incidental that it's a rectangle. I mean it could be a hypercube. Whatever.
CRAIG: Arguably it is if you consider tabs where you can have formulas on one tab, so it's not strictly rectangular, right? It goes beyond that. I think you can even pull in data from other sources.
ALAN: Right.
CRAIG: Anyway, we're kind of going outside the useful bit.
ALAN: Right.
CRAIG: I'll throw it back to you.
ALAN: Yeah. No, but you're absolutely right. As a computing environment, the average desktop spreadsheet, it takes on variables and namespaces. It takes on dependency or propagation of new values throughout the thing. The part that we talk about when we say spreadsheet is the part that propagates the value.
Now, in ClojureScript, when you're using this javelin library, there's nothing grid-like about it. Your values, your input cells are going to be named either locals or top level definitions, just like any other ClojureScript program. And what javelin does is it gives you a way to connect those places in a way similar to the way that you connect cells in a spreadsheet. So you can have the same expectations about what data will appear where when these inputs change.
You're welcome to make a grid of these if you want. It's pretty straightforward to make a spreadsheet when you have javelin, and probably even easier now that there's ClojureScript, and ClojureScript is pretty easily available. You can very easily make a spreadsheet in ClojureScript, a ClojureScript spreadsheet. We haven't. I don't think we've done that. I'm not sure anyone has done that, but that's because we already have awesome spreadsheets.
MICHA: But, yeah, some of the things that are interesting about a spreadsheet is that first you have two kinds of cells. There are input cells and formula cells. Input cells are the ones that you edit.
ALAN: Right.
MICHA: A formula cell is like in Excel. You type equal and then you type a formula. That formula is like an in – I can't think of this word. It's really killing me. Invariate?
ALAN: Invariant?
MICHA: Invariant. Yes. Thank you.
ALAN: Yes, or a constraint.
MICHA: Right. Right, so this is like an invariant. A formula is an invariant. You specify that if you have a formula, so let's say you have two cells: A1 and B1 in Excel. You're just going to type numbers into that. Then you have cell C1, which is =A1+B1. That says that cell will never be its value. The value associated with that cell will never not equal the sum of the value in A plus B. Additionally, there is a guarantee that the formula will update only once. In other words, if you make one – when you change input cells, formulas will not update more than once or unnecessarily.
If the value – if the things – if the cells that they depend on have not changed their values, no updating will occur, so work does not need to be done. And if work does need to be done, it'll be performed only once. This is pretty important because you can imagine sort of a naïve way to make a spreadsheet would be to, you know, you have your cells. And whenever anything changes–kind of like how macro expansion works in Clojure–you keep evaluating all the formulas until they stop changing.
Originally I had a simple thing that worked like this. We kind of devised – that's called glitches in, like, spreadsheet parlance kind of – so glitch elimination is important because you have situations like we called it the pregnancy test, which was, say you have a form. And the form has your sex, which could be male or female. Then, if you're female, you could be pregnant or not. Every time you click in this – and so this is in, like, a user interface in a form. Any time you modify this form, it should send the current state of the form to the backend, but such that it is always valid. Meaning, if you're male, you're never pregnant and so on.
If you don't have glitch elimination, it's pretty much impossible to do this without building glitch elimination into your system because, suppose you're female and pregnant and you change your sex to male. It has to know to uncheck the female part. Sorry, the pregnant part. With javelin, we make that a formula cell and stuff.
ALAN: Cells together in javelin can constitute a type, like a ClojureScript def type. But even cells in a normal spreadsheet, they have a contract, which is, every cell has a value at all times.
MICHA: Yep.
ALAN: And that maps up really well with the way Clojure works and Clojure's philosophy of state and time because, in Clojure, there's a rigorous separation of looking at something versus changing it. Like every reference type in Clojure, you have the ability to de-reference it. You can see its value at a point in time.
MICHA: And to add a watch.
ALAN: Right. So you can – these reference types, atoms are the most popular, but there are agents, refs, various–
CRAIG: Agents, vars.
ALAN: Yep.
MICHA: Yeah, var. Yeah, all of these things. You can see their value and hold onto that value for the rest of the program. You don't need to worry about what happens to the underlying container, the box. And javelin cells work like that too. Both the input cell and formula cell types support de-reference. And this de-reference, because we do the propagation eagerly and in dependency order, the de-reference is going to give you a consistent view.
CRAIG: Mm-hmm. Yeah, so in working with javelin, it was actually one of the things that got me using hoplon for the little side project I've been using it for is that, you know, I'm a backend guy. I don't really do the front end stuff very much, although I've been doing more of it recently. Kind of it's always a question of when you roll up to a new library or framework, what do you need to know?
For me, to a first approximation, the thing I needed to know in order to understand javelin, which is an important part of doing a hoplon application, was atoms and watches, which are very familiar concepts to me as a backend programmer. Now, of course there's more, right? You guys were talking about the fact that you can have chains of dependencies, and there is an atomicity guarantee. But to a first order, it seemed very comfortable. It's like, oh, there's a thing and I can look at it, and there is an API for changing its value, which is actually swap and reset. Hey, that looks a lot like an atom, so that part was all very familiar.
MICHA: Right.
CRAIG: I think the stuff that you were saying about the model makes a lot of sense. I do want to make sure that we talk as well about hoplon because javelin is very cool, and I found it a very comfortable, familiar, easy to acquire way to model my programming, not without some onboarding, but it didn't take very long. But, you know, it wasn't bad. I think most people would roll up to it, so I'm curious then to talk more about – well, I don't want to cut you off. If there's more interesting to say, we should say it.
ALAN: Yeah. No, I see the way you want to go, and it's a good direction.
CRAIG: Yeah, yeah.
ALAN: Yeah, so javelin has nothing to do with the browser or the DOM or HTML, which is part of the reason I think that we feel like it's done as a library. It occupies a part of the hoplon framework and our workflow in which it's done. We're not going to add anything to it, and we're not going to deprecate anything. It's done as a library. Which also, if that was all there was, it would be useless because, like, how do you make a webpage with it?
CRAIG: Right.
ALAN: Well, there is no inherent way in doing that, so there is a linkage, conceptual and software, between javelin and the browser. That is traditionally we call that hlisp, and there are library components of it that are in the hoplon boot task, but it is a set of conventions for working with the native DOM elements that allows you to use native DOM elements to provide views into your javelin spreadsheets.
I don't know. Does that seem like a good–?
MICHA: Yeah. I mean hoplon originally, like in 2012, doing stuff in ClojureScript was a lot more difficult than it is now. And so a lot of this stuff that is hoplon was to make that easier too. Then when we made boot, we incorporated them into the boot tasks.
Hoplon itself is a library, a ClojureScript library that you can use in any application. It doesn't need boot or anything like that. But the boot task does things like creating the actual HTML file in which you're going to run your JavaScript. You know, pre-rendering stuff, which means you can run your application in PhantomJS to generate the HTML. Stuff like that.
There's hoplon the library and then there's the stuff that boot does, which is optional, but me myself, I find it useful. I don't want to be hand coding HTML files any more, so my applications are kind of more like just JavaScript that I want to launch into somebody's browser. The only way I can do that is if an HTML file is generated somehow.
Hoplon itself, it's sort of meant to be a foundation for something more useful, to write something more useful on top of it. It doesn't actually do very much at all. It just allows you to wire up javelin cells to DOM elements. You can think of DOM elements as, well, first of all in hoplon, all of the DOM elements that appear on the user's screen in the browser are created via JavaScript. Meaning they're JavaScript objects first, and then they go into the DOM. They're not HTML. They're not DOM elements that are created from HTML.
When you think about it that way, the goal of it is to provide a platform that you could hook up your application state, which is expressed as these cells, these formula cells, input cells, and so on, to widgets that you make by composing these DOM elements. And so, like, if you think about a JavaScript, the JavaScript representation of a DOM element, you have properties and methods on them. You can call set attribute and so on.
My kind of intuitive view, the way I think about is attributes are like methods on the DOM element. Kind of like a world line of this, of the method, so if you take like the class attribute of an element and you hook it up to a javelin cell that might contain different values over time, meaning the class might change over time, you could sort of consider that to be a method that's called every time the value in the cell changes.
ALAN: It would be as if, say, you had the class string in an atom. Then separately you created a DOM object. Then you made a watch, an add watch with a function that set the new value on the thing every time it changed.
MICHA: That's precisely what hoplon is actually doing.
ALAN: Right.
MICHA: It just makes it easy for you to express it as attributes when you type your source code.
CRAIG: Right. In Alan's example the nice convenience is that you don't have to link up the cell or the atom and the attribute via some piece of code that runs. You just say the value of this attribute is the javelin cell, and then the wiring happens.
ALAN: Right. Conceptually, you can think of attributes as continuous, basically.
CRAIG: Hmm.
MICHA: Yep.
ALAN: As opposed to one shot things that you have to set manually.
CRAIG: Mm-hmm.
MICHA: Yeah. I think of it like a world line of a method or something. You have this object that has a method, and it's going to be called with different values over time.
ALAN: Right. But you don't care about that plumbing, really.
MICHA: Right. The thing that represents the values, which will be used over time is the cell.
CRAIG: Mm-hmm.
ALAN: Yep. Yep.
MICHA: That's what – so in hoplon you have the hoplon element, these DOM elements. They have attributes, which are like methods on the object, and they have children. Children have a special relationship in the browser so elements can contain other elements, obviously. And so the children can also change over time. That's really it. There's not much more. Hoplon, what we wanted to do with it is we wanted to make a foundation that we could build UI kits on top of.
ALAN: This comes back to the original motivation with the white labeling with the Fresh Diet.
MICHA: Right.
ALAN: This was kind of – we've arrived at the problem we, well, set out to solve originally, which was how do we describe an application in terms of both its workflows and its UI in separate pieces such that we can modify one or the other without necessarily modifying the other.
MICHA: Yeah, and to be able to – so the goal, the end goal would be that you identify some widget that's going to be useful to you. Say – let me think of – just take an input, like a text input.
ALAN: Auto-completing.
MICHA: Yeah.
CRAIG: I actually have an example from a side project that I think might be what you're talking about, and you can tell me whether you think it's–
ALAN: Yeah.
CRAIG: If you like, I can share and you can tell me whether you think this is what you're talking about.
MICHA: Absolutely.
CRAIG: I used hoplon to write a weather generator for my – my own hobby is flight sims. I'm making weather for this game that I fly in. It's random. It has neat features. You can control things about it. It doesn't matter.
I fly with other people online, and I would like the weather to be a surprise. Right? I don't want them to know that when we come back to the air base that they're going to have to land in the middle of a thunderstorm necessarily, right? But at the same time, you know, we're interested in this experience where you can see what the weather is because, hey, they could stick their head out the window when they take off, right, and they kind of know. And so I'd like to provide them with a weather forecast.
I have the same data, like I had this model that I can manipulate. I have a set of controls for doing that so I can step forward in time and say, okay, yep. The storm is going to arrive at 3 o'clock. That's about when we'll be landing. But then I want to also give them that same exact data, like the whole model is parametric, and so everything is completely deterministic. But I don't want them to be able to go past takeoff time because, if it were the real world, they would know what the weather was, but they wouldn't know what it's going to be.
MICHA: Mm-hmm.
CRAIG: And so it's the same data, the same computation, but it's a different view. It has limitations. It's not the same. If you were looking at it without knowing anything about the internals, it's not the same application even though the underlying data is. And not only data, but computation is identical. Is that kind of what you guys are talking about?
MICHA: Yeah, I think that's – well, what's interesting about that is that there's a separation of the presentation from the sort of state machine that describes the different things that get presented in the context of your widget, right?
CRAIG: I didn't quite follow that. Explain further, please.
MICHA: Well, so you have a widget there that is going to display the weather for the user, but they won't be able to see the weather for the future, right?
CRAIG: I've got two sets of users. One is me, the weather designer. I've got tons of controls where I can say it's 30% likely to rain. It's like the winds are this, blah, blah, blah, blah. Then I'm like, great. I know the weather for all time, including the future.
Then I want to be able to say, take that same data, some other user, and get a different view on the same information, but that's limited in terms of how they can interact with it. They can't change things. Maybe they can change a few things like where they are in time, but even there maybe not past a certain point that I specify.
ALAN: Right.
CRAIG: I was just really wondering if that's kind of what you were getting at around saying, well, there's still a workflow. There's still, when time goes from 2 o'clock to 3 o'clock, the following things change in the underlying. Like the data is the same, but the way I interact with that, what I can see, what I can interact with is different.
MICHA: Yeah, that's actually a good example because I could imagine the situation when maybe you are going from your view to, like, the pilot view also or something like that. And so you could be turning things on and off inside of your widget, like certain controls are only visible if you are in God mode versus pilot mode or something like that.
CRAIG: Mm-hmm.
MICHA: And so you might design a widget that has those that exposes them as attributes. You could have an external state machine that tells it which state it's in. The presentation part, the widget, the thing that's in the DOM would be controlled maybe – you know, the different configurations of it are via attributes, just like in the normal DOM elements, you know. You change the configuration of the built in DOM elements by adjusting the values of these attributes.
The idea in hoplon was to expose the same exact interface. You can make your own widget that exposes the same ways, means of controls, like these attributes and children, you know, that you could do the same thing on yours. The components that you make are not different than the ones that come in the browser like, you know, text area or whatever.
ALAN: Yeah. I think your example is great from another perspective too, which is you have this underlying process, which in your case is parametric and very well defined. But the magic to it, the purpose for its being is that not all of the users can see into all of it. Not everybody has full fidelity. Different kinds of users have different levels of access, kind of.
I can imagine in your case, you have the God mode view and then you have the pilot view. Let's say you have a widget for each of these, or maybe a set of javelin cell values that determine what certain widgets show or don't. Like I'm imagining your God mode thing you have a slider from the past into the future by some number. The pilot has a slider from the past to the present. That's managed. Whether or not the user can see the weather in the future depends on whether the God mode cell is true or false.
But maybe the God mode cell is not really that. Maybe it's named like the user level cell, and it can be one of God, you know keywords here, keyword "God," keyword "pilot," or maybe keyword "ECWACS," you know. What are those planes with the big saucer?
MICHA: Mm-hmm.
CRAIG: The AWACS.
ALAN: The AWACS. The AWACS weather guy has more fidelity than the pilot, but not as much as God. Maybe his maps are higher quality.
MICHA: He's a lesser god.
ALAN: Yes, a demigod. But we find the exact same thing in our business applications that we build with this stuff, which is, there are all these different concerns, and they're usually oriented around the way the person is supposed to interact with the system, whether it's constraints based on the kind of user they are or constraints based on what the way that's most efficient for them to work with the system is. Yeah, the separation of the widget behavior from the underlying model behavior is, I think, the way of building these things that gives you the most flexibility to do that. Especially, I mean, the key thing is being able to add, remove, and modify these perspectives over time.
MICHA: Yeah. The really key thing is that, well, I think, and we've pretty much achieved this with the application that we have at work is that there should be sort of two completely separate phases of development. Not even phases because they happen at the same time, but you have widgets that you're working on, so you develop these widgets. Then your application is actually just an assembly of widgets, which is different than – I mean it becomes an application. But by assembly I mean you only use composition, essentially, to construct it.
ALAN: Mm-hmm.
MICHA: So you make all these components, and the way you compose them – the way you build an application from all of these separate components is exclusively composition. I mean obviously there's going to be, like, a few little conditionals here and there, but if you were to look at the source for, you know, the Adzerk UI, all of the views are simple composition of these components. The big advantage there is, like, if you consider for example – like we did a lot of work around forms and how to handle forms. By that I mean things that talk to the backend and also interact with the user. We develop a really – I think it's a really good system where we have a state machine that describes talking to the backend. And so we have this RPC framework that we use.
ALAN: Castra.
MICHA: Right.
ALAN: The library, ClojureScript, Clojure.
MICHA: Yeah, and so the state machine on the front end is an object. It's a def record, and there are methods on this object that you can call, which might cause it to change state. It might not. But you can call them to command a change, and it exposes its state as javelin cells.
If you think about, like, you have, say, a login form is pretty simple. You have username and password. There will be – you construct a form machine for that form and maybe tell it the schema so it knows you have these two fields. But then it'll expose to you an error cell for the username and an error cell for the password. And it'll expose the data cell for the user name and the data cell for the password because there might be some default there or you might have already edited, whatever.
The machine has this. And you can wire those cells up to DOM elements on one end, and whenever you change them – so the way we like to do it – there are many different ways to make the state machine, but the one that we settled on is you can edit a form as much as you want as a user. When you first click submit, that's when validation happens. Then thereafter, as you type, the validation is done as you type. The validation is actually all done on the backend to simplify, to simplify our whole model. So every time you type, it's validating on the backend. And the error cells are being populated.
The error cells, or there's also like a general state for the state machine that is exposed to the cell. But the benefit here is imagine I'm actually making the widget now for this form. I can have a widget, a sub-widget that displays an error, right? I could have a widget that accepts input from the user. These things are completely decoupled from the state machine because they're going through this common interface now of attributes. So for example I might have–
ALAN: And cells.
MICHA: Right, and cells. So I might have, like – so we in fact do have this. We have our own text input, which does certain things and displays itself in certain ways. Whatever. So you do text input and then colon value, which is the way you set attributes, and you give it the data cell from the form machine. Now whenever you type in there, the form machine is getting that updated data because it also, you know, knows about this cell. Likewise, the error gets passed as an attribute to the little widget that we make that displays the errors.
CRAIG: Mm-hmm.
ALAN: And you know, I think, if people listening to this are coming from a JavaScript background, they'll recognize some of the affordances of this setup and the idea of a promise, which is a kind of trending JavaScript idea. But a promise, if you're not familiar, it's basically a value that you can get or an object that you can get that will contain a value. Clojure has them too, incidentally. And a value can be delivered to the promise once. When the value is delivered, there's an API for a function being called. It's kind of like a one shot atom. Where a promise is a one shot atom, a cell is like a continuous promise, again because it doesn't just fire once when the value changes. Whatever thing is looking at it, you can use stuff in javelin to make happen whenever the value changes, forever.
CRAIG: Mm-hmm.
MICHA: Yep.
ALAN: So–
MICHA: Well, the difference there, though, is that javelin does maintain the dependency graph.
ALAN: Yeah.
MICHA: So like if you have a web of promises–
ALAN: Right.
MICHA: –that's where you get into trouble.
ALAN: Right, right. Yeah, unless you're very diligent.
CRAIG: Yeah, and so like I said, I've built applications, and this makes sense to me. In fact, it was kind of the first UI framework that I tried that I did really connect with. And I mean, to be fair, I didn't give all of the others as much of a run as I've now given hoplon. But still–
ALAN: Oh, I know that you were a huge spreadsheet fan originally, too, right?
CRAIG: I do love spreadsheets, actually.
ALAN: I imagine we suckered you in a little with that.
CRAIG: Yeah. No, there's definitely – I am a huge fan of spreadsheets. However, there's an xkcd where there's like an amusing graph of complexity and, at the left end, is a two line bash script, and at the right end is some spreadsheet that's been being maintained for 20 years to maintain the scheduling on some church in Georgia, right? That's the joke in the slide in the comic. I'll have to post the link to that.
The point, though, is that there exists some extremely sophisticated spreadsheets in the universe. I think something that some of our audience will have been exposed to is this common task, which is, well, I've made a spreadsheet. Now the spreadsheet is inadequate, and so now I have to turn it into a "real program."
MICHA: Right.
CRAIG: I guess what I'm trying to ask is I could kind of imagine that the cell model where you have this potentially very wide graph of interconnected bits of computation to have a sort of complexity ceiling. Now I've only used hoplon to solve smallish problems. I mean I think the thing that I did actually is nontrivial. The things that I've done are nontrivial.
But, you know, they're small compared to anything that I would have done if I were working on it day in and day out like you have been doing with your work at Adzerk. Have you experienced limitations of the cell model? If I squint, I could convince myself that you would fall off a complexity cliff eventually there. Has that been your experience?
ALAN: I guess I would say there are two kinds of ceilings. One is easily navigable and the other is, I guess, sort of open – open, and we could talk about this thing called UI. But cells definitely have straightforward, technical limitations just because of the way the browser works. The foremost of these is that they're not automatically garbage collected by the JavaScript runtime because of the way we maintain the dependency graph and because JavaScript doesn't have a weak reference and probably never will for security reasons. It's possible using cells naïvely, and people do this regularly, to have memory leaks. And so there's a certain diligence that comes with working with cells, and I think it's easy to learn. It's, in practice, not a huge problem. But it's a definite hard technical limitation. You can easily drive yourself crazy trying to if you sort of have the wrong mindset. You can sort of correct the way that you're thinking about using cells and easily get yourself out of this.
The other limitation, which is kind of the open problem, is cells presume so little about – cells and hoplon presume so little about how you're going to build your application. Micha talked about this a little earlier when he said that, you know, our goal is to get you to the place where you can start to solve the problem. We don't presume to solve the problem completely for you at all. I think how you approach that is going to be application dependent, and whether or not you reached the complexity ceiling is kind of on you.
I don't know. Maybe this is a good time to talk about the UI library.
MICHA: Yeah, I mean–
CRAIG: I would love to hear about that, actually.
ALAN: Okay, or maybe Micha.
MICHA: Yeah, I mean I don't actually see – like, we have really big hoplon applications now. Big meaning like many, many different screens and a lot of complex workflows and back-ends. You know, so the one that we're building at Adzerk is a pretty complex application and complex workflows. We really haven't – like, I don't really see that we're even approaching any kind of – I don't see a complexity growing at all.
CRAIG: Hmm.
MICHA: The reason is because, just like in a spreadsheet – like, the reason why spreadsheets get unmanageable, I think, is mostly because, like, the names. In other words being in the grid with A1 and B7 and so on, like that's okay if it's small. But, you know, if you're making a bigger – if you're making an application with spreadsheets, you kind of want to have names for things.
ALAN: Right, and you can't have anonymous cells in a spreadsheet, of course.
MICHA: Right.
ALAN: You can't make self-contained machinery.
MICHA: Right.
ALAN: Everything is global.
MICHA: Yeah, so this isn't something you run into with javelin. Also, like with javelin, the formulas that you do have end up being pretty small. If you see a formula starting to get large, you just break it up into smaller ones, just like you would do in a normal spreadsheet. And I think that has – I mean, for me anyway, it just happens naturally that most of the formulas I have are very small, and they all just kind of mesh together. And because of the glitch elimination, it's not like – like, we don't even really have any debugging tools for javelin. You know? There's no – like Alan, I know, made the graph view of it. We keep losing it and so on. I just never really found it necessary.
ALAN: Well, see, I think that can still be a thing. I think that could be a thing. Some of the best criticism we've gotten about hoplon is from Daniel Higginbotham, a great friend of mine, awesome guy who worked at Adzerk part time on this hoplon system. He did point out that, you know, figuring out what's going on is less than easy if you're the people who made it. I think we can improve there, but we've gotten pretty far without having that.
What I imagine, in my copious free time, doing at some point is making a layer or an extension for Chrome that gives you a visual graph representation of the cells. Then you can click on the cells and see the source code for the cell or its value. I think that would be awesome. It's attainable now because we can put metadata on vars in ClojureScript. I think that's just a matter of time.
To Micha's point, it's a conceptually simple model. And Micha mentioned that you can split up a big cell into smaller cells, and you can do that because of our glitch elimination, because of the consistency properties of the model. You can basically, by rote, break a cell into a set of smaller cells, and you don't have to reason again about the states of these cells.
MICHA: Yeah. When I need to debug something, I just make a cell that has a print def in it that prints out the value of a cell, and I very quickly – I can pretty much bisect any weird javelin issue, I mean, and because the types of issues that you might have are actually computing problems rather than timing issues. You know what I mean?
CRAIG: Hmm.
MICHA: They're not likely to be race conditions like you would have if you were using a bunch of promises.
ALAN: Right.
MICHA: Like a web of promises.
ALAN: Right.
MICHA: Right, so these are all, like, you're just getting the wrong value in some cell. So tracking down how that value arrived there is way simpler than saying, like, is it getting the wrong value because things happened in the wrong order?
ALAN: Right.
MICHA: That's the kind of thing that's, like, time consuming.
CRAIG: Yeah, absolutely.
MICHA: For the next step, there's this thing called hoplon UI. Jessie has been working on it. He's been doing a ton of work on it. His goal there is to make the next level foundation, which provides basically a sane and well factored platform for building user interfaces as opposed to, like, documents.
ALAN: Basically a new browser. He's building – it's basically a browser API on top of javelin and the conventions in hoplon that we've established. Like where today if you take the stock hoplon stuff and you make an application, you're interacting more or less with stock CSS stuff, stock JavaScript script things, DOM things, which is great because that means there are really no surprises. You can stack overflow, search for things.
CRAIG: Yes. I've done so much of that.
ALAN: Yeah. Yeah, so it's all the native stuff, so it's Googleable. But you're not really escaping the inherent complexity of the browser environment by doing that. You're choosing to live in it, for better or worse.
The idea with UI, as I understand it, is Jessie has been using hoplon for years, almost as long as we have, and built a series of applications, and identified in his own work patterns and practices. He has put a lot of effort into codifying those into this library UI, which is a higher level starting point that doesn't make you give up too much about being in the browser, if anything. So the fidelity is still there, but the complexity is reduced, which is – I mean that's a big value proposition. But I think it's really impressive work. Micha probably knows more about how it works and stuff than I do.
CRAIG: Okay.
MICHA: Yeah, I mean, like an interesting thing that you can do with hoplon is, instead of using CSS in CSS style sheets or, you know, style tags, whatever, you can manipulate attributes from cells to set the properties directly.
CRAIG: Yes. That's actually super interesting that you can do stuff with that that you really can't do easily any other way.
MICHA: Right, so like even SCSS or LESS or Sass, those things are computing CSS statically and emitting a CSS file. But with hoplon, you can have a javelin cell that represents the styles for your element that updates via ClojureScript at runtime. Meaning, constantly it's a first class. It gives you first class access. What Jessie has done is, on top of that, he's using that to rebuild. So he's had experience doing Flex and all kinds of action script stuff.
ALAN: AIR? Is that one of the things?
MICHA: Yeah. He's done like a ton of basically all the different ways you can make user interfaces. From that he's sort of collected a core set of functionality. One of the main things is that, in the DOM, you have documents and user interfaces. The two are not really very well separated. There's a lot of concerns for, like, making a newspaper that are conflicting with concerns for making–
ALAN: Gmail.
MICHA: Gmail. Exactly. And so that's really kind of what he's attacking. He's attacking the user interface side.
CRAIG: Hmm.
ALAN: Another example of a simplification he made, which is one of those, like, makes total sense in retrospect things, and it's just we're constantly fighting with since we started working with JavaScript seriously is the fact that there are so many different kinds of elements in HTML.
CRAIG: Mm-hmm.
ALAN: Like <div>, <p>. Now there's <section>, <header>. These are the concerns of the newspaper imposing themselves on the concerns of the people who make things like Gmail in 2016. It's insane. And they all have different default styling attributes and behaviors and different flow properties, have different implications on the positioning model, and so what sold me on the UI concept was he just has a single kind of thing, the LM, ELEM function. It's a general purpose object that can represent a piece of the user interface.
CRAIG: Hmm.
ALAN: You can put them inside each other. They all have the same number and kind of attributes. They all have the same positioning and style, default style properties. Yeah, that's just an example of the….
MICHA: Yeah, and some of the stuff that – so it's also a low level library in that it doesn't necessarily give you the components that you will actually use to build an application. It provides the underlying primitive that you need to build a user interface. And so the kinds of things that it does is make an element that fills the screen and put inside of it children who fill its parent, who fill their parent, but in a certain way. Meaning like the middle one should be 10%, the one on the left should be 20% of the width, and the one on the right should just fill the rest of it. Or you might have the one on the right actually contains two others that themselves are each 50% of the remainder.
CRAIG: Mm-hmm.
ALAN: Right.
MICHA: Things like that, so it's essentially – you know, it's kind of ParEdit, but there's one type. So there's the LM, but there are also special form. The different form elements, like an input box, things like that, those need to be special cases just because of all the concerns of–
CRAIG: Sure.
MICHA: –user interaction like when the persons have completes and all that stuff.
ALAN: Right.
MICHA: So you want those to be – a lot of times the browser, like, you need to choose a particular element. You can't make your own.
ALAN: Right.
MICHA: But–
ALAN: But he investigated doing that. I mean I remember him talking about his early work where he thought, you know, what if we do the browser visually from first principles? Why can't I just use SVGs for everything?
MICHA: Right.
ALAN: You know, whatever.
CRAIG: It's interesting you say that. I was thinking about this, and I would say the things that I've been gravitating towards as I've been building my applications are div, SVG, and flexbox, which actually sounds like a poor man's version of what you're talking about.
ALAN: Yep.
MICHA: Yep. That's exactly what Jessie did as well.
ALAN: Yeah, so just do that for five years–
MICHA: Right.
ALAN: –and you'll have UI.
CRAIG: Let me get started. I gotta go.
MICHA: Yeah, yeah. He did a full–
ALAN: You're on the bat, yeah.
MICHA: He has a lot of interesting stuff to say about flexbox. I remember when he was like – yeah, he made everything out of divs for a while.
ALAN: Yep.
MICHA: Yep.
ALAN: I mean designers frequently do the same thing. I mean for a long time a hallmark of any serious, large, single page app was the reset.css. It's like, okay, let's turn off everything in the browser so we have a place to start with that's predictable.
MICHA: Yep. So what's interesting about UI is that this LM, this generic component is actually three – it's constructed from three divs. There's like an outer one, a middle one, and an inner one. Each one of them has its own role to play because of the way padding and margin and width and so on work in the browser. He realized that you need three of them in order to get a consistent – in order to be able to consistently–
ALAN: And to satisfy every aspect of the–
MICHA: Right.
ALAN: –positioning and styling model that he wanted to support, which is his view of how you should do it.
CRAIG: Hmm.
MICHA: Right, and so you would think that you'd end up with like a ton of three times as many components in your application. But in fact, I think we're finding that there are way fewer elements compared to, say, a bootstrap. You know, like bootstrap, the Twitter bootstrap is one way that you can – you know these CSS frameworks–
CRAIG: Sure.
MICHA: –that are sort of trying to accomplish the same thing, give you a kit, a UI kit that you can use to construct your application from. But they actually end up with, like, a lot more elements because kind of the only way they can fix things is by adding more structure because I think, fundamentally, like the LM that Jessie has developed is a really good, fundamental unit that's worked out at a low level.
ALAN: It's the ultimate div.
MICHA: Right, so, like, in order to fix some weird padding issue in a bootstrap thing, you'd have to add more wrappers, you know, like maybe a clear div, whatever.
CRAIG: Yep.
MICHA: You're like adding more and more stuff. Then that compounds itself when you compose those with other and so on. It's really very interesting. I think that we're finally reaching an age, like in web, you know, the way people use the web that a lot of business stakeholders are not going to – they don't care about, like, inventing some new user interface concept. We've already figured out how to interact with the user to get information and give information to them. What's really important, I think, to people now is the workflow. In other words, if somebody comes onto my application, they're like a business user and they want to actually get something done and their time is precious, they don't care if it's all just gray. You know? Nobody cares what it looks like any more. Nobody is going to be impressed with, like, some fancy drop shadow or something. They want to know that you can get in, do their work reliably, and get out.
ALAN: Well, that's the big difference between these. This is kind of the dual, the two faces of the web. And they're always at odds because I don't think people recognize them as totally different concerns. The newspaper, like are you going to make a killer above the fold, catch people's eye brochure site, or are you going to make an application that people are going to live in for eight hours a day? Within the first 30 minutes of serious use, they're not going to even notice what the colors of the things are. And the primary concern is usability, not necessarily flashiness.
Sorry to cut in. I just–
MICHA: Yeah, totally.
ALAN: But, yeah, that's kind of – like in the world that we seem to live in, building large sites with JavaScript, we're definitely on the user experience side, not the new media or design side. Granted there is still some give and take there, but for the most part people seem to be interested in what Micha said: the workflow, like, is this an efficient place for someone to live and work in all day long, like one does in Gmail?
MICHA: Yeah, so I can totally imagine something built on hoplon UI, which implements all of the – like I'm pretty sure that we can now enumerate all of the types of widgets we're going to need to make business applications. And we can make, you know, instances of them, and we can also make, you know, sort of a zoo of state machines that can be composed to form the really responsive workflows that you need, the really efficient workflows. In other words, the workflows that help the user accomplish their task more directly.
CRAIG: Well, you guys just blew my mind. I definitely have to check out hoplon UI. And, you know–
ALAN: Oh, but an important caveat.
CRAIG: Yeah, please.
ALAN: Jessie says we shouldn't use it yet.
CRAIG: Okay. Cool.
ALAN: But go ahead and use it.
MICHA: Yeah. Yeah, everybody has been using it.
ALAN: Yeah.
MICHA: And he has, like, 10,000 warnings.
ALAN: Yeah, the Read Me is just warning after warning, you know, this is unstable. Don't use it. But of course there are businesses using it now, so.
CRAIG: Yeah, well, I could kind of imagine that I've got completely broken, 10% versions of what he's already doing, and so his 90% semi-broken version might be a vast improvement for what I'm trying to do.
ALAN: Well, in my case I used it as a reference or a Rosetta Stone to figure out how to do a CSS thing. He's got the distilled knowledge in there, so there was some CSS3 feature I needed to use on some side project recently, and I was able to figure out how to do it from his code, so it's useful there too.
CRAIG: Well, guys, I don't want to cut off the conversation because I think – in fact, I know we could keep going, and I for one would be sitting here fascinated, but I think there may be people in our audience that need to use the restroom.
ALAN: Okay. Well, they can wait for just one more little point I want to make.
CRAIG: Absolutely. Absolutely.
ALAN: Which is, another hoplon community guy, Thomas Herman, he gave a screencast, which I'm really – I feel bad that we didn't record because it was incredible. But he demonstrated the power of working entirely in code instead of trying to straddle document and code in terms of making an application. He's a heavy Cursive user. Colin Fleming, another Clojure community superstar, has this tool: Cursive. It's a great Clojure and ClojureScript environment for IntelliJ.
I don't use it personally, but I have a lot of friends that do and they love it. And it turns out that when everything in your application is ClojureScript code, then you can start to use your ClojureScript IDE as a design tool.
CRAIG: Mmm.
ALAN: Thomas Herman is building a site using UI and, in his demo, screencast, he like hovered over the attributes in one of these LM things, and the Cursive tool tip popped up with all the color values that it could take at that place in the code because this was, you know, code inference that incidentally was style inference because all the style is happening in code. And, you know, he was defining sets of colors and defining maps of class names to colors that Cursive was smart enough to track down and show in the editor, which seemed like a quick little peek into the future of the design tools.
CRAIG: Oh, yeah. I mean I know this is only a corner of what you're talking about, but I totally want now to go back and put into my little apps that CSS code because the means of abstraction there are garbage. And I'm not using Less or Sass, right, but it's just terrible, right? You have to say the same thing six times, and it's static, and blah, blah, blah. And so I think even just that little piece of what you're talking about, to me, looking at it as a front end newb, I'm like, man, that seems like an obvious win. Why don't we have that normally?
ALAN: Yeah. Yeah, and you know, I guess I wouldn't go so far as to say that Less and Sass are bad.
CRAIG: No. Sorry. That's not what I was saying.
ALAN: Yeah. No, no.
CRAIG: I was saying I'm not using them, but CSS itself lacks abstraction.
ALAN: Right. They're great for brochure sites. They're great for designers who are making media. But for programmers, not so good.
CRAIG: I see what you're saying now. Okay. Yeah, I gotcha. Yeah.
ALAN: But, yeah. Okay. I'm done with my thing.
CRAIG: Cool.
ALAN: You may go to the bathroom.
CRAIG: Yes. It is a podcast. There's a pause button.
ALAN: That's true.
CRAIG: Anyway, yeah.
ALAN: It didn't occur to me.
CRAIG: No, but I do think it probably would make sense to wind down.
MICHA: You can just hit mute.
CRAIG: There you go.
MICHA: You'll be fine.
CRAIG: Mute my part. Mute my part. I'm not saying anything interesting. But I do want to kind of bring it to a close here, and I want to make sure that we have – I mean, Alan, obviously you said you wanted one more thing, and that thing was very worth hearing. I'd like to make sure we give Micha the same opportunity. Micha, is there anything else that you think we should touch on before we wind down to the final question?
MICHA: No, I think we're good.
CRAIG: Cool. Yeah, this was awesome, guys. Yeah, I really enjoyed our last conversation. And, as I was coming to this one today, I'm like, okay, cool. We talked about boot. We'll talk about javelin and hoplon. That'll be good. That'll be good. Clearly there is a lot to talk about here, a lot of really interesting, really important stuff to say, so much so that I will feel comfortable in saying that we will have you back on again.
I don't think we'll do a third part to the show, but at some point in the future, the not too distant future, we'll have you back on. It'd certainly be great to hear more about what's happening in the hoplon, boot, javelin, Adzerk, Alan, Micha universe. Always good stuff happening there.
But I do of course have one more question for Micha, which is, we'd like to close the show with a piece of advice. I'll go ahead and say that use hoplon is now obvious. Maybe that was your piece of advice, or at least try it, I suppose, would be the better way to put that. But I'm imagining you have something else in mind. What advice would you like to share with our audience today, Micha?
MICHA: Can it be like a little pro tip kind of thing?
CRAIG: Literally anything you like.
MICHA: Oh, well, I really enjoy – I have a paintbrush in my bag, and I use it to clean off my monitor and my keyboard constantly. Not obsessively, but just whenever there is stuff on my keyboard, and I really enjoy it.
CRAIG: Hmm. So it'll get, like, between the keys, all the crumbs and everything that wind up in there or whatever?
MICHA: Totally!
CRAIG: Huh.
MICHA: Yeah.
ALAN: Well, at one point you compared yourself to an umpire.
MICHA: Right.
ALAN: Kind of a similar habit.
MICHA: Yeah, sweeping off the plate, getting ready to go.
ALAN: Right.
MICHA: And I only mention it because now there's like three paint brushes around the office or something.
ALAN: Yeah.
MICHA: Generally, people enjoy it, so maybe, you know–
ALAN: Yeah, everyone seems to crack their knuckles.
MICHA: Right.
ALAN: Pull out their paintbrush. Dust off their screen and keyboard. And then–
CRAIG: Well, I have to say I will take that one to heart because I have been known to be on the phone with somebody and to kind of flip my keyboard over and bang it on the desk. As a result, like, drop it on the floor and hang up a call. And so I think the paintbrush is a much, much better way to handle that. I like that a lot.
ALAN: Yeah. Elegant.
CRAIG: All right. Well, look, guys. Thanks a ton for coming back on. I'm actually really psyched that we were able to do this so soon, at least in podcast time, after the last show. I think they complement each other really, really well, and I've been really into your tech. I said last show, but I'll say it again. Thanks for making this stuff. It's really cool, and it's made me feel incredibly productive in the browser space that I have avoided for, like, since 1991 when I first saw a browser, so thanks a lot for both that and for coming on the show to talk to us about it today.
MICHA: Sure. Thank you.
ALAN: And I said this, I think, the last time we talked about hoplon, but we have a really small, really helpful community, mostly in Slack, in the Clojurian Slack. If you find yourself toying with this stuff and are stymied or just are curious to see what other people are doing, feel free to join us. We're always happy to help and talk about what we're up to.
CRAIG: Awesome. Good tip. All right, guys. We will call it an episode. Thanks so much for being on. This has been The Cognicast.
[Music: "Thumbs Up (for Rock N' Roll)" by Kill the Noise and Feed Me]
CRAIG: You have been listening to The Cognicast. The Cognicast is a production of Cognitect, Inc. Cognitect are the makers of Datomic, and we provide consulting services around it, Clojure, and a host of other technologies to businesses ranging from the smallest startups to the Fortune 50. You can find us on the Web at cognitect.com and on Twitter, @Cognitect. You can subscribe to The Cognicast, listen to past episodes, and view cover art, show notes, and episode transcripts at our home on the Web, cognitect.com/podcast. You can contact the show by tweeting @Cognicast or by emailing us at podcast@cognitect.com.
Our guests today were Alan Dipert, on Twitter, @AlanDipert, and Micha Niskin, on Twitter, @MichaNiskin. Episode cover art is by Michael Parenteau, audio production by Russ Olsen and Daemian Mack. The Cognicast is produced by Kim Foster. Our theme music is Thumbs Up (for Rock N' Roll) by Kill the Noise with Feed Me. I'm your host, Craig Andera. Thanks for listening.