Episode Archives

Improving Code Reviews with Github’s Copilot

How does GitHub Copilot and Codespaces help data scientists to write, understand, and review code?

Do not punish learning in software engineering teams

What does it take to foster a workplace culture where employees, specifically coders, have the liberty to learn without feeling punished for it by the system? Innovation is impossible without failure, but most work cultures suffocate creativity without realizing it.

In this episode, I talk to Dr. Cat Hicks, a data scientist, a behavioral scientist, and a creative entrepreneur.

We talk about:

  • how she deviated away from a traditional path of a researcher to start her company, Catharsis Consulting, 
  • how to foster a learning culture within your engineering team
  • what learning debt is and 
  • how learning debt hinders software engineering teams to reach their full potential. 
Dr. Cat Hicks

Today’s episode is sponsored by Codiga, a smart coding assistant and automated code review platform. Try Codiga for FREE!

Subscribe on iTunes, Spotify, Google, Deezer, or via RSS.

Transcript: Foster a learning culture in engineering teams

[If you want, you can help make the transcript better, and improve the podcast’s accessibility via Github. I’m happy to lend a hand to help you get started with pull requests, and open source work.]

Dr. McKayla 00:03 Hello, and welcome to the Software Engineering Unlocked Podcast. I’m your hosT Dr. McKayla, and today I have the pleasure to talk to Dr. Cat Hicks. But before we start, let me tell you about an amazing startup that is sponsoring this episode Codiga. Codiga is a code analysis platform that automates the boring parts of code reviews and lets you merge with confidence on GitHub, GitLab and BitBucket. I’ve worked with Codiga for around one year now, and I love how it guides me in discovering the, well, not so nice parts of my code base. But there is more: Codiga also has a coding assistant that helps you write better code faster. Find and share safe and reusable blocks of code within your favorite IDE on demand while you are coding. Codiga has a great free plan, and so there is nothing that actually stops you from giving it a try. Learn more at Codiga.io. That is Codiga.io.

But now back to Cat. Cat, or Catherine Hicks holds a PhD in experimental psychology and is a principal researcher in Team Lead at catalysis consulting. She has designed researchers at places like Google Khan Academy and co founded a startup that builds tools for software engineers, and she led multi institutional collaborations in online learning. So I’m super, super, super thrilled to have a cat here with me, Cat, welcome to the show.

Dr. Cat Hicks 01:30 Thank you so much. I’m really excited to be here.

Dr. McKayla 01:33 Yeah, me too. I’m following you for a long time now on on Twitter. And I was very impressed because you have a similar not the same, obviously very different. But you have a similar background, like coming from academia and then going independent. And and so yeah, so it was very interesting to see how you built Catharsis Consulting. And you’re the founder of Catharsis Consulting, right?

Dr. Cat Hicks 01:59 That’s correct.

Dr. McKayla 02:00 How did it help people with empirical research and also empirical research? I really a software engineering area, and you are now empirical researcher coming from experimental. How do you help companies?

Dr. Cat Hicks 02:14 That’s right. That’s right. So it’s delightful to connect. I think there are a growing cohort of us out there, you know, in the world who have made this journey, and there’s not really a roadmap for us. So I’m always I love to talk about it.

Dr. Cat Hicks 02:27 I like to call catharsis and evidence-science consultancy. So this means that we help partners use evidence to inform their decision making and tell their stories. And in particular, we’re very focused on meaningful measurement. So I describe it to people as not just data for the sake of data, but creating research methods that give us data that’s fit for purpose. So I try to help partners who are trying to learn something real about the world they’re working in, and how to move forward. And we have a couple

areas of competency and special focus. But we’ve led projects, it’s easier to give examples right, then talk high level. So some recent projects are, I’ve asked things like how can we find evidence that a product design change in a language learning game actually increased the learning that was happening for children using the game. Another recent project is using surveys to help a small nonprofit tell the stories of how community members that they worked with, were helping people in their own families and in their social networks, learn about the COVID vaccine and make the decision to try to get that vaccine. So both of those projects, very different scale for those projects, very different types of data. But both of those projects connected to really immediate impact, whether it was on product design, or on an intervention, and programming that help doctors have better communication with their patients. So at catharsis, you know, we try to bring a few core principles to all of our research projects. One of them is just that people deserve to understand their data, and to really use the data that they already have maybe special access to, and to try to bring the tools of empirical research to everyone, even small organizations that may not have invested in learning those skills before. So one thing that we bring in a lot of our partnerships is an emphasis on teaching those research methods and taking it not just from, you know, findings on one project right now, but actually fitting any work that you do with data into a larger plan of moving forward.

Dr. McKayla 04:39 Yeah, and it sounds really, really exciting. And it reminds me, I’m more on the training side, right. So I’m helping a lot of software engineers actually get better at code reviews, but all of that also based on empirical research that I did around code reviews, but recently this year, actually half of the year I spent on I’m a research project with this startup and help them come up with a framework on, you know, what makes developers developer experience really great. And they created a product out of that. So I can totally relate to that. And it was really a wonderful experience. But the experimental nature of research and startup somehow there was also a lot of tension, I would say, right, there was a lot of, you know, it’s very different than in an academic world, you have like these questions, and here was, every day, I want to squash the question, is this impactful? Is this impactful? Right? I don’t know if you experienced that as well. And how do you handle that in your work?

Dr. Cat Hicks 05:42 I think we all experience right, right, you know, there’s a tension between things that you’re doing for the long term and, and needs that are in the short term, and then, you know, just to be really real about it, I think there’s a lot of people who have agendas about what we’re going to find. And yeah. And I, you know, I try very hard to always work with partners who, who have told me, you know, we are going to make changes based on what we find, even if the changes are uncomfortable to us for even if it it, it helped, we learned something that conflicts what we thought was before, this is very important for social impact work, you know, is very important for equity, when you’re going to do anything that has to do with people’s well being, but it is a core tension. And I think that researchers, we tend to be people who love the truth, right? And we’re just all about finding out the truth. And that can ruffle feathers. I love to do exactly what you described, where you go from working closely with people who are living an experience, and then translate that, you know, to leaders and to organizational structures. And I think it’s a beautiful role to be in, but it requires a lot of invisible work, right of explaining both sides to each other.

Dr. McKayla 07:00 Yeah, and, and working with this tension, right, which I think, is tell for me, it was a very challenging time at a time that I learned a lot. Because I’m, you know, for me, the rigor of the methodology is the most important thing. Right? And, and then comes time for the sort of, it’s more the time and then the rigor I think, right? Like, yeah, obviously, you know, like, at least a little bit, there is the priority or they are more stressed by a timing, then you know, then a researcher probably, yeah, and so on. So you have to deal with these tensions. And I think it was a very, very interesting learning experience for me. But what I really love this, that I could see, this research transformed into a product. And this was, this was actually the reason also why I loved academia because I was missing that. Yeah, getting real, right, I created a lot of prototypes in my in my research career, and actually think some of them maybe would even have had some potential even for open source, right, maybe not making tons of money, but some open source software that people would have used. But it was never the time. Again, we are coming back to time, but in a different way. Right. So the time was up after the paper is published, the time was up to work on that. And so I felt like I couldn’t really translate it into what I would like to see. Right. And that’s why I left for example, academia. How is that for you? Why did you leave this traditional path of a researcher and and start your own company and do your own thing go independent? Right?

Dr. Cat Hicks 08:40 Yeah, for sure. So you know, I think that it’s interesting because I am a researcher who likes to study environments. So whenever you ask someone about their choice as an individual, I think you have to see it also as a choice about what was around them. So I’ll be uh, you know, I’ll be real about that. I mean, academia is very hard to succeed in, not, not because of the quality of your work, but because of the opportunities that are around. And I but I think that there was a really core piece of what I loved. So I started out working in classrooms, I started working, asking about the beginning of how we learn to learn and even in my academic work, I was very interested in being in real schools talking to real children was where I started I did a dissertation with 3 to 11 year olds, so you can imagine Oh, yeah, yeah, asking young children about their how they were thinking about mistakes and how they were thinking about learning. So from the very beginning, so cool, you know, it’s amazing how much it pays off, right? Because we all we all start there and even now I work with adults, you know, and, and yet, all of the same questions come up all the time. So, you know, I think getting I found it beautiful and amazing that people are constantly scanning around them asking whether it’s okay to make mistakes and asking who they can talk to. And I just, you know, I saw a lot of exciting stuff out there in in tech. I think the journey for me too, there’s a personal, you know, that’s kind of the problem space. But being an entrepreneur is also a way for me to carve out this role that I did not see existing. So I always felt a little bit like, I’m a social scientist and a data scientist. I’m a data scientist, who cares, you know, about how we measure things. I like meaningful data more than big data. You know, it felt like with catharsis, it was a way to make the job that I wanted to have, you know, to do these kinds of projects.

Dr. McKayla 10:47 Yeah, that’s exactly what I did as well. I love create the job that I would like to do that, then that I feel like I can strive it.

Dr. Cat Hicks 10:57

And it takes courage. Yeah, no, you have to have to say, I know this is valuable, which I think you do as a researcher, just like you were talking about that startup, sometimes you have to be the person who’s saying, I know that this will pay off if you will do it, you know, you haven’t measured it. So you can’t see it yet. But I know it will. Because I’ve been there working with people and I see their pain and frustration or whatever else. And then they build it into a product. Right. And it does pay off.

Dr. McKayla 11:23 Yeah, exactly. Right. Yeah. So I looked at your newest report, which was super interesting for me, because it is around the software engineering teams. And there you shed light on the learning debt that we have, and how that can affect engineering teams. Can you tell us a little bit more about what this report is about what this software? Or what is research actually investigated? Or looked at? And what is what is learning data? And why do we have it as software engineers?

Dr. Cat Hicks 11:56 Yeah, great question. So as a part of Catharsis’ work, I can occasionally invest in this sort of work basically, for the field. So this is a report I did, because I found it really interesting, and shared publicly, and it’s called coding in the dark. I interviewed 25 software engineers or developers, and I asked them to share about their active problem solving as they were ramping up on an unfamiliar codebase. So this was people talking about their real jobs right now. They shared about code review, they shared about how they asked for help, how they collaborated. And I’ve shared a lot about, you know, what we talked about. And essentially, you know, what I found was even at these really big tech companies, most of the people I was talking to, we’re all at big tech companies. Even at these places, people’s experiences were really quite frustrating. So I called this report coding in the dark, because that was a quote from one of the people I interviewed, describing how they felt every day, like they were showing up, and the lights were all off, you know, and that they were having to fumble their way through learning without any help from anybody. And there was this core tension that they experienced between feeling like it was so important to learn to build their understanding, to experiment, iterate, but then when they showed up, you know, to code review, and to other moments where they were being evaluated, that learning was not being valued. So I described this cycle, you know, of needing to do this work, and then finding it devalued. And going back to your kind of heads down at your desk, you know, I describe that as learning debt. And learning is essentially the dynamic that happens when people know they need to put a lot of effort into learning. And they know that the kind of work they need to do requires these mistakes. And it requires this long term understanding. And there’s kind of all of this stuff that you’re doing that’s sort of invisible, because it’s not showing up in your productivity. And they also know that the environment around them is only measuring that short term productivity. So in this kind of environment, where there’s a lot of learning, debt, accumulating, essentially, you know, learning you have to be do that you’re not getting rewarded for there’s also a lot of performance, pressure, and what’s worse, you know, things like documentation, writing code, comments, trying to help other people, you can actually feel actively punished for doing that. Another quote in the one of the interviews I led was that learning would be seen as a waste of time. And I think one of the engineers called documentation and code comments, a red flag about your abilities as an engineer. So you can imagine how that feels. You’re in an environment that’s telling you to do all this complex work, but also telling you that it’s a waste of time if you help anybody else. Learn from what you’ve learned. So, you know, a big conclusion that I have in this report is this debt cycle this learning debt cycle can accumulate

damage for a long time because teams might look very productive on the surface, but you’re building what’s really an inefficient experience for learning. So I’ll stop there and kind of Yes, more.

Dr. McKayla 15:09 So yeah. Tons of question for now, the first one is really what kind of persona did you interview? You were saying people that are new to a code base, but it is, are you? Did you ask them when they were onboarding? And is that the onboarding experience for people? Or is that somebody that’s already on a team, but within you problem, it sounds more like an onboarding, experience. And, and, and heavy onboarding experience. But

Dr. Cat Hicks 15:37 yeah, it was a mix, it was a mix. So I think that one thing that’s interesting is that you might think, Oh, this is somebody who’s just new to a whole company, you know, they’re experiencing the, but actually, I found this was a repeating cycle. So some people were fairly junior, you’ll see there’s a, there’s a cross section of seniority. So we really wanted it is a qualitative project. So it’s, it’s not intended to be a representative sample, I think, you know, follow up surveys on this kind of thing would be really, really fun to work on. But in this cross section, we did have a good number of junior folks, but also senior folks, even a couple people who are leading the engineering teams at their organization, I did ask them to bring in an exam, think about before the interviews, when they were a recent problem they had of basically trying to understand someone else’s code. So if this for some people, this was a really brand new codebase, right, like the whole thing as they were joining a company, but for some people, it was just a piece that they hadn’t really touched before. So yeah, it was happening really all over the place. Right? Yeah. So the

Dr. McKayla 16:48 other question that I had, when I tried to envision this is, what kind of learning because there are many things that are you know, that we can learn as software engineers, and I feel that everything that has to do with technology is rewarded, and is seen as something, you know, that that you get some credit for at least, and that it’s also very internally, a lot of engineers like to learn new technology. But then if you’re coming to code basis, the main knowledge, right, all the work that you have to do to understand this piece of code for code review, and so on, right? I can maybe relate more than this is the kind of learning where would see this, you’re, you’re supposed to already know it. Right? So let’s skip that step. You know it, and then you do your productive work? And why do you you know, this is somehow the invisible thing? Is that is that, you know, is my just guess, here? Is it going in the right direction? Or what kind of learning? Did you? Did you investigate here?

Dr. Cat Hicks 17:48 Absolutely. And I love that you have called out the complexity of learning. And you know, it’s learning is a big word for a lot of different things. Right. And, of course, you’ve had some really phenomenal thinkers who have broke out on this podcast that I’ve really enjoyed, who’ve talked about, you know, productivity is not one thing, satisfactions not one thing, the same could be said for learning. So, you know, I, I thought a useful contribution in this report would be to talk very broadly about the beliefs we have about learning. But the actual specific examples are a lot of different things. And I think it does map on right to, to exactly what you said. So developers feel like, Oh, if I’m learning a new language, or

a new piece of, you know, a new tool, something that’s very explicit, right, that’s easier to defend. And it’s easier to justify. But the focus is always on the technology, right? And the production and not so much on, oh, now I really understand how this other team has a mental model of, you know, this connection piece, or I really understand this dependency that happens. And I understand these trade offs.

Dr. Cat Hicks 18:55 So you know, there’s actually a tremendous amount of content I got in these interviews, that’s not even in the report, because it was so much. I think it could be another report on the kind of active learning that they were doing. And a lot of it felt, you know, almost secretive, like people were saying, oh, you know, I’m sure no one else has to do this. Like, I do have to go back and remind themselves, you know, because I don’t want to talk about it, because I’m afraid I won’t look like an engineer. But the reality was, to me a lot of that stuff, like thinking about the trade offs of different decisions you made thinking about whether a design decision, you know, that we put on paper really was that way in the code and even questions that are kind of like, is it worth the investment to fix this inefficient piece when I could instead be working on this other piece? You know, these are very abstract things for people to be thinking and learning about but they’re really, really critical. And I was reminded to, there’s a lot of myths around learning, right? And as a social scientist, I recognize some of these myths. So people will tend to think, once I learned something, it’s just learned forever, right? It just goes into like, my brain is a bucket, and I just dumped something in there. And it’s always gonna be there. But actually learning is really a behavior over time. So the mourn environment cannot see it as shameful, but see it as beautiful and productive and great that sometimes we’re asking each other for help. We’re reminding ourselves how things work, you know, and you see that when developers talk about googling for answers, right, and asked on Stack Overflow, and all of these other kinds of things that people do. But it was interesting to me how much they hid that stuff from their environment. Yeah,

Dr. McKayla 20:44 yeah. Because the real engineer knows all the keyboard shortcuts.And I think it’s so it’s so true, what you say, right? So learning what is learning? And if we are making a decision around trade offs, I think very often it’s not framed as learning. And then it’s also how, you know if I can write it down. And you know, if it’s not in a book, if it’s a very specific instance of something, another general thing that I can learn. What does this even mean? Right? So we learn, for example, about object orientation, and you know, how to how to have objects, but then to really think about this piece here. And the instance of should I create an object here? And how should the object look like and that I have to think about that is a little bit shameful, because, obviously, I learned object oriented programming. And so it should be easily coming to me, you know, what methods I should put in here, or naming, right? naming a method? Yeah, it’s also learning somehow, or we have to put the time into, and then it’s hard. And even though we make jokes about it, if somebody sits next to you, and you have to think about a good name, and only stupid names come to your mind. It’s horrible.

Dr. Cat Hicks 22:07 Yeah, and I think you’re, you’re, you’re pointing out something that’s actually really, really important here, which is, you know, there are good jokes and bad jokes, right. And we’ve, we’ve been around, we’ve all probably been around someone who has made a joke, you know, that has made us feel really

bad about how we learned or a mistake that we made. And this is something that came up in the report to that, you know, I think one of the quotes was, I’m always watching, like, I’m from a junior code writer, or someone said, I’m always watching the senior members of my team, because I want to know, what an engineer is supposed to sound like. And that can be really beneficial. If the people around you are saying things like, we all make mistakes, we all forget something, you know, we all help each other. That’s a good learning culture. But a negative learning culture, right? A bad culture is a place where people are, are saying, oh, you know, don’t waste your time, like doing this documentation. Like in order to get ahead, what you actually need to make sure you’re doing is putting on this performance. You know, things are very multifaceted, as you know, all of these things are always happening at once. But I do think that there’s in engineering culture, there’s a lot of myths around what brilliance looks like. And this is where I’ve pulled from some research from people like Andre Symbian, who’s done some work on, you know, when a field thinks that you don’t make mistakes, you have to just be born brilliant, then that is a story. That is not how it works, right. But we’re all kind of upholding that myth.

Dr. McKayla 23:41 Because we all want to be the 10x Engineer, right? And then we had to have 10x engineer. Oh, my God. Yeah. No, but something.

Dr. Cat Hicks 23:51 Yeah, something that, you know, just just hurts my heart, honestly, to is is like, the people who people do this work, right. People do mentor other people, they do support learning. And that actually is what creates 10x results. It I mean, investing in learning is one of the most evidence backed ways that we have to you need to do work together. And I had, if we could see it as something that we are sharing, and that we’re all working on outside of ourselves, you know, it’s it’s never about, you write bad code, I write bad code. All right, fine. Like we work to make the code better. It’s outside of us. And it does not tell me who you are as an engineer. In fact, a good engineer is someone who’s written a lot. Yeah, I mean, we need to, you know, we need to improve things and give feedback, right. But I think we need to value the messages that that feedback sends.

Dr. McKayla 24:47 Yeah, I think that I want to come back to this different kinds of things that we learn and, you know, writing good code, whatever that means. And it’s also I think, changing over time. What is good code, right, whatever, what is a good way to write code but a good applications, how to structure them that also evolves? But again, I would say this is this textbook knowledge, right? And then I think what’s, and this comes back to code reviews and to the data day to day work that we have to do and to productivity a lot as well, is this constant learning? Right? I cannot stop learning. I cannot, you know, it’s not like, Oh, now I work. You know, obviously, you get better at this code base, and more familiar with the terminology and with your, how your team works, and so on. Yes, right. But still, even if I’m at this team for three years, and have worked with this code base for X years, right? If I have a new change that somebody else wrote, then I have to look at this code. Yeah, starts right there. And you know, and I cannot come in and have this full bucket of knowledge of how that works. And then, you know, supposed to already point out what was going wrong here. And maybe it has to do with how we measure time, and then a lot of people I think, have really problems with time, I have a lot of problems with time, like, when when should I leave the house to be on time, right? And I think very similar, we

estimate, for example, how long will it take to make to look at this code and give comments. And very often people reduce that to the time to make the comments, but this learning part that is never stopping continues and will have been added nobody wants to talk about and you know, nobody actually wants to have and nobody has time for it. That’s that somehow gets forgotten or is forgotten. Right? Yeah. Oh, I

Dr. Cat Hicks 26:42 agree. I agree. And I time came up a lot in these interviews. And it doesn’t surprise me because, you know, we all have felt this time pressure. And what I kept asking was, you know, if you’re experiencing this time, pressure, like, what is the the first thing that gets cut? is, honestly, to me some of the most valuable stuff, and that is really hard for people. So, you know, there is a sense in which I think I totally agree that doing this work, the learning will never stop. And you’ll you know, it can feel a little overwhelming. But I think that that’s a reason to say, you know, what success is not you getting to the end of your learning. Like that’s not what success is success is having enough space to make a good decision instead of a bad decision about how we move forward. And I, I did see people go through that. And actually, you know, I agree that it’s very difficult sometimes with this work to predict how much time it’s going to take. And I experienced that with my own work. People ask you to do a research project. And you say, okay, like, it sounds, it all sounds good. But I need to get in there and see what the truth is. And we might learn, it’s way more complicated. So I think about things like, you know, can we have measurements of productivity, that is dynamic, that we’re able to come back to and change it, and I think people will get get very, very frustrated, you know, when they are assigned a project, they dive into it, they do all this learning, actually mapping out how complicated it is, is a very valuable piece of learning that they’ve done, and they turn around and they want to share that with somebody, and there’s no way to share that, you know, there’s no way to kind of get credit for it. So that’s, you know, that’s one thing I think about is if we can make some of that more visible, right, like, like, allow you to use the learning and share it with collaborators, I think that people really enjoy that they feel the productivity of it, even if your goals of the project change. Another thing is, you know, can we talk about where time pressure makes sense? And where it doesn’t make sense, right? So can we prioritize and and see the cost of putting everyone under a time crunch all the time? And where that is just creating these learning cycles? Yeah. So

Dr. McKayla 29:06 what I want to understand a little bit more is, there were definitely some outcomes from this report, tell us how prove right? How can we reduce these learning that how can we have this growth mindset? How can we, you know, how can we in our at least in our engineering team, a celebrate learning and make it a bigger priority? What are some of those outcomes? What can you suggest engineering teams that want to improve their learning experience? And, and the valuing of that?

Dr. Cat Hicks 29:40 I think there’s a piece of this puzzle for every different role, right? So, you know, from leaders, from engineering leaders, these people could have a really outsized impact on the culture and I think that you know, a lot of places will put a poster on the wall that says everyone could learn or or maybe there’s a bullet point in a slideshow about like we’re alerting culture. But if you go to work and you see someone actually get rewarded for a complicated learning situation like, hey, you know, we gave, we

told you to go try to do this thing in the codebase, it turned out the thing we, you know, the thing that we proposed was not possible to do. But you did all this learning, you figured out a better way forward, we’re gonna celebrate that instead of, you know, coming down on somebody for it being not what we expected, those kinds of moments. And I think leaders have the ability, you know, to, to notice that to try to push themselves to amplify that that can have an impact. Another thing I would suggest, you know, that I suggest in the report is, we honestly need to separate some of our development feedback from some of our performance feedback. So okay, I don’t know how many conversations you’ve had with engineering friends, about perf cycles. But perf cycles are a huge source of stress. And even though we have invested, this is a whole area of research this ton of people, you know, who look at this, but even though we have invested huge structures into it in tech companies, a thing that I keep seeing as a learning scientist, is that we are rarely letting people have psychological safety to talk about them learning. So I think that a very simple step that leaders could take, is to make space to separate when you’re talking about how you want to learn and grow and develop and maybe explore areas of growth for you. And separate that from promotion, performance, reputation management times that you are trying to defend yourself, which is very difficult, you know, you can’t really do those two things. At the same time. I have a number of other recommendations in the report, you know, I think that there are some simple steps like, have we put any time in our calendar for documentation? Or are we just acting like that’s gonna happen magically by itself? You know, so there are small and big steps to try to make yourself a learning culture. Does that all make sense?

Dr. McKayla 32:05 Yeah, totally. And I think documentation again, is, maybe it’s the last thing that I want to talk a little bit about. Because I think there again, we have these two different kind of learnings of information of sharing. So you have this external documentation of how things work, right? And, and people agreed, and you know, in API needs documentation, but then the nitty gritty part becomes a little bit translucent, right? It’s like, oh, this method, actually, you should be able to understand it just by looking at the code. Otherwise, the code is not good. And don’t put a comment there. That’s really bad. Right? And I, I, sometimes I, I really can’t understand the problem here. Because while it’s great, if you know, and there are different learning types, and you know, in different people that maybe somebody is easier, you know, it’s easier for them to look at the code and really get it then skip the skip the comment, right? And some people like the comment, and it gives them context. And you can really know in, in, you know, native in your native language or in in, you know, in written language instead of code. But again, here, there comes this the myth a little bit as well, right? We say, well, code shouldn’t actually be documented. And you shouldn’t need documentation to read this. And there is also some research around that. And they showed that if there are comments in the code, people are slower with reading the code. But why? Because they are reading the comments. Right? And would they read the comments if the comments are useless? No, they are reading the comments, because they’re actually helpful. Right? And

Dr. Cat Hicks 33:45 that’s such a good example. Yeah, that’s such a good example of a measure that like is taken to be a negative measure. But why it might actually be a positive measure? Yeah. Yeah, I think it’s, you know, you bring so much rich lived experience on this, and I love hearing it because it’s, the reality is that these are going to be contextual decisions, like a code that was as you said, code that was good,

quote, code, quote, unquote, in one time, my deal if the context has changed, and then that then, you know, you need to make a different decision. And I think that there’s, there’s there were these interesting quotes, you know, when I interviewed people about who, who is this for? Is the documentation actually, for me? Or is it for, you know, like, some idealized scenario where we’re describing the technology and point of view that I have in my consulting, you know, is that I like to focus on people as the heart, you know, and that code writers as learners, like if we, if we take this approach, where we center they’re learning, we can be a lot less afraid of things like sometimes the trade off is that you have more comments and that that doesn’t work for all situations, but you have preferred did some really deep losses in efficiency and invisible losses that are happening? So if someone’s able to ramp up a lot more quickly, that’s a huge game. And I think something difficult about it is that sometimes that gain is really invisible. But you know, it’s, it’s not really possible to have a single way of describing code that’s going to work for everyone who’s ever learning. Yeah. And similar to measuring developer productivity, I think it’s, it’s a question of what is the best thing for us right now. And what’s going to pay off the most, even if it slows us down a little bit in this way, then I think it will really pay off. If you know, later, this person who we gave all the support to is able to become this champion contributor. And I just think, you know, I use the learning debt cycle, like the learning debt metaphor to, to evoke tech debt, because we understand tech debt right in this field. And we understand that technologies with all these dependencies can start to break apart, even if it made sense when we built it. And I think the same is true for collaboration. Yeah,

Dr. McKayla 36:11 Yeah. Yeah, there’s so much goodness in that. And I really want to dig into the productivity. So maybe what I want to do is I’m going to invite your again, a whole episode just on productivity if you’re up for it. Yeah. And then we can really dissect that, because I would love to hear your, your opinion also on, you know, you mentioned or hinted a little bit towards that. Can we measure learning as part of our productivity? Right. And I had a podcast where was just me talking about productivity. And there, I was asking the question, I was saying that all these productivity measure that we have focused around activity, right, coming from an area of the industrial age, right, where, well, it was the activity that better Yeah, exactly. And it was that the activity that we did, right, you had only to do very mechanical tasks, and the small task, and so you could count them, and so on. And all those measurements actually stem from there. And now we put them on knowledge workers, were probably the most productive thing is that I’m sitting here doing nothing, but I make a really good trade off this session. Right? That’s right,

Dr. Cat Hicks 37:21 that’s right, or you help someone else and they do something, I would love to have that conversation. And I do think there are ways we can measure learning. And you know, if anyone is going to be listening to this, like, go to your team right now and ask, what are the things that we do that really make a difference, that are not being captured anywhere that are not being rewarded? Like what is the stuff that you know, is important to do to keep this all of this running? And they will tell you?

Dr. McKayla 37:53 Yeah, yeah. And coming back to what you say, with the sharing, I think what I have seen work really well is small things like brown bags, right? Where we come together, and somebody just explains what they have learned this week. Or if you go back to code reviews, right, that you that every Friday, for

example, that’s happening on GitHub every Friday, they are sharing, and sometimes they are sharing, what did I learn this code review? That was really excellent. Right? You know, it’s a comment that, uh, no person really took the time and gave me great comment. Or I’m showing some code that I have seen that I haven’t seen before, or, you know, some some Yeah, paradigm or something that I’ve seen. So and we are sharing, we’re making some of those very implicit things that are internal that are not, we are making them explicit and sharing them. And I think this is a celebration, as you said, I think those are themes can maybe do to celebrate what’s also often referred to as blue work, right? Oh, this, this colleague helped me or that person, you know, they didn’t work on their ticket, which had them for their promotion, but they actually went out of their way and did this and that, right. And so, we open openly sharing this and making it explicit. And I think, especially in our remote world now is more important, right, that we have shared that somehow. Yeah, no, I

Dr. Cat Hicks 39:16 think that that’s a beautiful point. And really, really important. And, and something that I also thought, you know, something again, that people people said in the interviews, which was, I want to see specific examples. I want to sit next to somebody and see them, see them code, you know, and, and that just I think people don’t know how much that doesn’t happen. I think they assume it’s happening or they say, oh, go get coffee, you know, with this person who wrote the code, you’ll, you know, go talk to them. But people often struggle, especially if you’re remote, you know, if you’re new person, there’s all kinds of ways in which people you know, reasons that people might not ask for help. And as I told you, I started out my career looking at Three and five year olds and in classrooms and when they ask for help, and even when we are four and five years old, we’re looking at the people around us. And we’re asking, Can I Can I ask for help? Can I talk to you about my real learning? So that continues? And the more you see those small messages, and those small social moments can just have a huge impact.

Dr. McKayla 40:23 Yeah, yeah. And I think team culture and psychological safety, and all of that is so important. And it’s, it’s, it’s not something that you can just fix by doing three things today, right? It’s something that you can start. But it’s a continuous process. And I think this is one of those very rewarding things and you know, things that pay off, but are a little bit invisible, that you have to constantly work on that right, and that you have to raise the bar and say, we are actually allowed to have questions be wrong, you know, growth mindset, and I think it’s really, it’s a continuous work in a team, but the teams that managed to do it, they are so much better off than That’s right.

Dr. Cat Hicks 41:07 And it just is a beautiful part of it, you know, I try to make these problems easier for myself, and for other people by saying, who’s already doing this, right? Like, how do we give them a stage to do it? Like, you’re who’s the person that someone always everybody goes to this person to ask for help? You know, how do we make sure that they are instead of being like, burdened by this invisible work, they’re actually rewarded for all this support that they’re doing? Yeah.

Dr. McKayla 41:34 Yeah, that’s so true. Well, it kinda, it actually brings us to the end of this show. I said, I’m going to bring you back. If you have time. I will continue. We can continue this discussion a little bit more. But is there

anything that you want to tell my listeners, maybe that you think, you know, wraps up some of the learnings that would be powerful for them for the software engineering teams? How can they, you know, be in a better place? What are what is the one advice, you know, that you would give them?

Dr. Cat Hicks 42:08 Yeah, great question. How, what a lovely question to be asked, you know, I think I end the report that I recently released, saying, learning matters. And I would I would like to leave with that, which is that, you know, learning matters and measurement matters. Like whenever we measure something, I think, Who is this measurement for? And is it bringing us closer to this culture that we want to have, you know, where we feel free and happy and, and like, we’re all learning together, which is what we need in order to tackle these huge, complicated problems in the world, you know, we need to get past some of these myths about where brilliance comes from, and the myths that we all need to hide, you know, are learning from each other. But that people will only be able to do that if we make the environment around them safe. You know, so it kind of comes from both sides from from us building the environment as individuals in it, but also from people who are able to kind of say, well, I’m gonna, I’m going to do something to make this environment safer. So that’s what I would say, you know, learning matters, it pays off. Let’s let’s work for it.

Dr. McKayla 43:18 Yeah, that’s beautiful. That’s really great. So thank you so much cat for being on my show. And I will definitely ping you again and ask you for more of your input. Thank you so much. Okay. Bye bye.

Dr. McKayla 43:35 This was another episode of the Software Engineering Unlocked podcast. If you enjoyed the episode, please help me spread the word about the podcast, send episode to a friend via email, Twitter, LinkedIn, Bell, whatever messaging system you use, or give it a positive review on your favorite podcasting platforms such as Spotify or iTunes. This would mean really a lot to me. So thank you for listening. Don’t forget to subscribe and I will talk to you in two weeks. Bye.

 

Running a developer community

In this episode, I talk to Bekah Weigel, who runs the virtual coffee community about community building. 

Bekah graduated from a Bootcamp in 2019 and quickly created a striving and very special developer community in just under two years. 

We talk about:

  • how she kick-started the developer community virtual coffee
  • what it takes to run the community
  • how sponsorships make it possible to be sustainable, and
  • how community members take over a large part of running the community. 
Bekah Weigel

Today’s episode is sponsored by Codiga, a smart coding assistant and automated code review platform. Try Codiga for FREE!

Subscribe on iTunes, Spotify, Google, Deezer, or via RSS.

Transcript: Kickstarting and running a developer community

[If you want, you can help make the transcript better, and improve the podcast’s accessibility via Github. I’m happy to lend a hand to help you get started with pull requests, and open source work.]

[00:00:00] Michaela: Hello, and welcome to the software engineering unlocked podcast. I’m your host, Dr. Michaela. And today I have the pleasure to talk to Bekah Hawrot Weigel, a web developer and creator of the virtual coffee developer community.

But before I start, let me tell you about an amazing startup that is sponsoring today’s episode: Codiga.

Codiga is a code analysis platform that automates the boring part of code reviews and lets you merge with confidence on GitHub, GitLab and Bitbucket. I’ve worked with Codiga for around one year now and I really love how it guides me in discovering and improving, well, the not so nice parts of my codebase.

But there is more. Codiga has a coding assistant that helps you write better code faster. Find and share safe and reusable blocks of code within your favorite IDE on demand while you’re coding. Codiga has a great free plan, so there’s nothing that stops you from giving it a try today. Learn more at Codiga.io. That is Codiga.io.

But now back to Bekah. Bekah graduated from the bootcamp Flatiron school in May, 2019. And since then she started a consultancy specializing in front end development and created the developer community virtual coffee. She also recently started her new job as a technical community builder at deep gram.

She’s also a mom of four, so I’m totally impressed. And yesterday I went to pick her brain on how she could develop this awesome. Develop a community so fast in just a little bit under two years.

So that come to my show background, I’m really, really excited that

[00:01:41] Bekah: you are here. Thanks so much for having me. I’m very excited to be here. Yeah.

[00:01:45] Michaela: So can you tell me a little bit about virtual coffee, what it is? And for me it seems a little bit different than other communities. It seems a little bit, a little bit more niche, grit, like closer.

W how would you describe

[00:01:57] Bekah: it? Yeah, I think that’s a great way to [00:02:00] describe it. We always like to say that we like the intimacy of virtual coffee because we’re a small community of developers where all stages of the journey. So if you’re just learning, if you’ve been doing it your whole career we’ve got everybody and we’re tech agnostic, so it doesn’t matter what, what tech tools you’re using.

If you want to meet up with other developers and share and support each other. We’re here for it. So we meet up twice a week when we meet up on Tuesdays at 9:00 AM, Eastern and Wednesdays at 12:00 PM Eastern for some chats. So we go into breakout rooms. So we have small group conversation. We like to maintain that intimacy.

And then for members, so people who have attended at least one virtual coffee, they’re welcome into our slack and our members only. So we have lunch and learns on most Fridays, we’re running our third round of lightening talks soon. We’ve got monthly challenges and some other small groups that meet that are building within the slack community, which is just so great to see everybody supporting each other and working to meet the needs of the community.

[00:03:04] Michaela: Yeah. So there’s a bunch of things that you just mentioned. Right? So a virtual coffee. When I came to know it, it was mainly there’s. The weekly, or I don’t know if it was even PVT at the start, but it was like this virtual coffees where you. We’re seeing each other and chatting to each other and now it grew into something really big.

Right. And so you say you it’s, it’s a small community, but, but how large is it? Like how many people are participating here and, you know, , what else do you do to keep this , intimacy, Ronaldo and Messi. Yeah, exactly. Yeah. How do you, how.

[00:03:43] Bekah: , Well, you know, we’ve got our slack has almost 600 people in it, but I would like to just note that I think that, you know, there’s a lot of people who used to be active that aren’t active anymore.

And one of the things about the community is we’re really close, but it is transient in nature [00:04:00] because sometimes people are looking for their first job and they get it and they can’t come around much anymore. Sometimes people change jobs and their availability changes.

So, you know, one of the things that we really like is being able to celebrate wins with other people.

It’s bittersweet a lot of times because you know that they won’t be around much anymore, but you know, like occasionally I’ll get messages from people who came at the very beginning and it’s just so great to, you know, still have that connection and know that, you know, we support each other, whether we’re in slack or not.

So I would say maybe we have about 200, 200 or 300 active members, which I think is still pretty good. 30 to 50% of our slack is fairly active. And I think, you know, We maintain that intimacy by doing, by focusing on small group conversation and a lot of ways. So the small group conversation that happens when we meet up on zoom twice a week we try and keep our breakout rooms between eight and 14 people, but we have.

Start in the big zoom room together. And we go over some announcements, like our code of conduct and what the mission of virtual coffee is a little bit of our history just to allow people to get a glimpse of this is how long we’ve been doing things. And this is how, how things have grown. And so. By having that interaction with other people by seeing faces or hearing voices or interacting and a synchronous way, it provides some kind of connection and friendship that doesn’t happen as easily in async only environments.

And then it’s great to see what the community members are doing. We have let’s see here, we’ve got all of these small groups meeting within the community and. Tech interview study group. These are all led by members that happened on Monday. We have an indie hackers meetup on Wednesday, a react meetup on Wednesday and [00:06:00] a monthly challenge check-in on Friday.

So, you know, the members are really there to support each other and to see what the needs are. And so not, everybody’s going to come to indie hackers. I like to go to that one. It’s one of my favorites. You know, maybe there’s like. To eight people there, but it’s great because you can really dive into those deeper conversations and get to know people in the, those small moments in ways that you can’t when you’re in a large group of people.

So I think that’s one of the things that, that we’ve done well is have people who care about each other and, and see them supporting each other in their new.

[00:06:39] Michaela: Yeah, that sounds really good. And there are a couple of things that I want to touch base on that you mentioned, first of all. So I can imagine that there are like a hundred people joining your soup, and then you have this announcement.

Remember you’re members of what is all about, which I think is really good for the mission also, and for new members to, you know, Just introduce them to your culture code of conduct and them so on, but then how do you announce the different breakout rooms? Do you know, do people speak up and say, oh, I want to do a breakout room.

Is it like, I don’t know if this is an a term in English by the bar camp where you, and non-conference where it just self-organize itself or do you have to announce it beforehand? Did you know already, these are the topics of, you know, today’s

[00:07:24] Bekah: Yeah. So that’s a really great question. So we also, I should have mentioned this before, because I think one of the ways that we also have been able to support everybody is we have documented most of our processes thoroughly.

And that allows us to bring new volunteers on and to support new people. We think of pretty much every interaction as a opportunity for onboarding new members and to constantly remind people of the things that matter to us, which is, you know, being kind and recognizing that the impact of our words matters.

And so we have all of that created and we [00:08:00] have let’s see maybe about 30 room leaders in note takers. And so we have a process on Mondays where we see who’s up for volunteering to be a room leader or note taker. And we pick a introduction question, just a random question. It can be something silly.

Like what kind of dinosaur would you be? And so everybody in the breakout rooms answers a couple of questions, including that. And we have a Backpocket topic, but we always say that we like to prioritize what folks who are in the room want to talk about. So if they have a question or if they have a topic they want to talk about, we start with that.

And then if not, we’ve got that back pocket topic. We had virtual coffee today and our back pocket topic. I’ll read it to you. Just so you have a sample of some of the things, actually, all of them I think are listed on our discussions in our repository. Our topic today was what are some transferable skills you bring to tech either from a previous career or from other parts of your life.

And so actually my breakout room did talk about that. And so everyone there’s okay. So we have the. MC he’ll gives all of the announcements. And then we have a host who controls zoom and the host puts everybody into breakout rooms. So we already know who our room leaders and note takers are. Those have already been set up the day before.

And so we make sure that they all get in room and we try and have backup ones. So, you know, if we need a fifth room, then we’re going to have this person as a backup room leader. Which today I think we did end up using our backup And then the host goes through and fills all of those rooms and we do our best.

It gets chaotic when people come in late or at the end or drop off and come back in. But we do our best to make sure that we get a pretty well-rounded room. So new folks, people who have been there for a while, you know that some people maybe are leading for the first time. And so you want to put some.

[00:10:00] Dependable talkers and their room. And so you try and make sure that you do that, but that’s kind of the process for what we do and how we do it. Wow,

[00:10:07] Michaela: that sounds like you’re having a conference and the organizational, like I say, not a burden, but burden ID. It doesn’t seem like it’s a burden to you to organizational fun.

Every week, twice sounds like a lot of work.

[00:10:22] Bekah: Now that we have the process down, it makes it a lot easier. And we’ve got some, you know, Slackbots reminding us about some of the things that we have to do and where to look for things. And it, it is a lot of fun. There is definitely work behind the scenes that happens to make sure that we have a, a safe and welcoming environment for everyone.

But, you know, it’s worth it. If people feel like this is a safe space that they can grow. Yeah, definitely.

[00:10:46] Michaela: And I can really see like with this organization, I mean probably if, if, you know, like if you started there like five people come in, you know, showing up, you don’t need a lot of organization. Right.

But then if 10 people, and then it’s 20, then you know, you started developing those processes and you probably see also what works and what does not work. And what are some of the things that you tried that didn’t work out. So.

[00:11:08] Bekah: Well, I will say that when I first started virtual coffee, I didn’t even know that zoom had breakout rooms.

So that was a totally new concept to me. And I feel like I’ve got some expertise in it. Initially. I so virtual coffee started. I had been working as a developer for about eight months when the pandemic hit. And then I lost my job because of the pandemic. My kids were sent home from school. That same day.

They never went back to school that year. I think. And so I was really interviewing for the first time for jobs and I just didn’t have a great sense of the developer community out there, what the expectations were, how to make it through the interview process. And so I asked, you know, Hey, does anybody want to meet up for virtual coffee on Twitter?

And, and that’s why we’re so called virtual coffee. And so I’ve learned so much. And [00:12:00] initially I was kind of resistant to having a slack because, oh, I don’t know if we need it. You know, this is just going to be something we do for a couple of months. So I, I would say that maybe some of the things that didn’t work were, you know, pushing some of those things off for a while or being resistant to.

Adding we, we lean on project boards and get hub issues a lot in our organization. We want to make sure that we use tools where our members lived. And so I initially I was resistant. I was like, I can not look at one more repository no more. And now I’m like, yes. Yeah, we need a repository for that.

So. I think that my, my, the thing that didn’t work was my frame of mind around it, because for a long time, I thought this was going to be a temporary thing. And when we did our first heck Tober Fest event two years ago, that’s when I finally thought, oh, this is, we’re not going anywhere. We’re w this is not just a pandemic thing.

Like we’re filling a need for a lot of people, even outside of the pandemic. And so that’s kind of where. Things started shifting in my mind, like, what is the, what are the long-term processes and how can we make this sustainable?

[00:13:12] Michaela: Yeah. And it’s really nice. Yeah. I’m also like, I, I’m often thinking of creating a community around se unlocked, for example, the podcast.

Right. But I’m not sure about how it will, you know, what are the right tools? What is the right kind of community. I’m also more a person of like, I’m not really good at. Participant in the slack channels and this core channel, I get very easily overwhelmed. And then, you know, like maybe a week I’m trying really hard, but

If it already starts with that, you know, I don’t, you know, I don’t think that I can run a community like this, but having, you know, chats or, you know, soon conversations. I was also thinking about Twitter spaces. Is that something that came to your mind that you could maybe do as well?

[00:13:57] Bekah: Yeah. So I [00:14:00] went through, I ran a lot of Twitter spaces myself.

I went through a string of them. I was doing them weekly, and then I started live streaming, I think instead just trying to get a feel for everything that’s out there. But I think with my job at deep gram, we’re going to start doing some Twitter spaces that I’m really excited about because of the support of the team.

And we can do some really great stuff. Start build community and fill some of the needs that we see out there in the tech community right now.

[00:14:31] Michaela: Yeah. Yeah. And do you think that there’s a difference between a zoom? It assumed seems a little bit more intimate for me because you know, it’s, it’s a community that’s not completely public it’s public, right.

Because people can just respond and be part of it, but to the spaces for me, just because it’s there on Twitter and then you see at least some of the bubbles and then it’s broadcasted through other, you know, sort of followers of the people that are in there. And so on. People can drop in and go out.

Do you think it’s different and, and has a different need or fills a different need, a different purpose for the condition?

[00:15:07] Bekah: I think, you know, there are expectations when you meet with other people in a small group setting, face-to-face, you know, you, and we say like, if you want to leave your camera off, if you want to stay muted, that’s totally fine.

If you want to throw things in the chat, that’s a great way to communicate as well. But still you see other people there, whereas Twitter spaces, you can kind of come in and out. You, there’s not the sense of, oh, I have a roll hill here that I have. Bill because you’re not a speaker. You can be a listener.

And so in. Twitter spaces I think is closer to watching a live stream because you can interact through the chat, but it’s a little bit more personal because if someone’s live streaming at Twitch on Twitch, you don’t see everybody who is there, but in Twitter spaces, you can see those other people and they do connect you.

Other like, you know, if we follow [00:16:00] each other, I can see whose space you’re in. I’m like, oh, okay, well, she’s there and she’s cool. So I’m going to go check out, you know, what she’s listening to. And so there’s a, I think a little bit, maybe more community happening in Twitter spaces, but there’s less like barrier to entry or friction if you’re shy or an introvert or, you know, just kind of want to check something out one time.

so

[00:16:24] Michaela: another thing that I wanted to talk with you, and I think they are a little bit connected. One is that, so I looked on your website, virtual coffee.io, and there are a couple of people publicly listed. Right. And apparently they are not all of them. Not everybody wants to be listed there or it doesn’t, it doesn’t.

Hasn’t edited themselves. But a couple of people are really listed there. And then I also saw that there are different roles, right? You were also talking about the different roles for the meetings, but there were two particular ones that were like labeled there. And one was the, the. Maintainer and the other was the community maintainer.

So what are those two roles? Is that all the roles that you have and how do you select people or how are people selecting themselves to be in those

[00:17:07] Bekah: roles? Yeah, that’s a really great question and it’s kind of evolved over time. We’ve had so many people step up and offer support and offer help. And Sarah, McCombs, they were really great support at the beginning of.

Virtual coffee and, and making sure that we got this stuff done and helping build out these processes. And when we launched our first hack Tober Fest, we had a whole team that was focused on that. And a number of the people who were on that team ended up coming on as maintainers and. I’m both a, a core maintainer and a community maintainer and, or a, an org maintainer rather.

And what that means is we kind of look at the overall organization, the health, the strategy where should we go from here? What decisions need to be made in terms of the entire organization? I would say the community maintainers are [00:18:00] looking more at the day-to-day, the community management project planning that, that kind of more day-to-day focus, I guess, in, in making sure that the team is supported there.

So we all work together as a core team and we make decisions together and there’s always going to be overlap in all of those things. But it’s funny that you ask about these roles because I was just working on this. Actually we have some team leads and they should be going up on the site there.

They’re already listed there, but. You know, we have leads for our monthly challenges as Areli, Varo, and Andrew Bush for our audio visual stuff. So getting things put up on YouTube, helping with live streams, that’s bogged in. For documentation, we just onboarded a new team lead named . Who’s absolutely great.

I met with her this morning to kind of like walk through the process of, you know, how do we prioritize, what needs documented, where do we put these things? And she’s so great about, you know, asking questions and getting issues up on the site before I even think about them. So I hope I didn’t miss anybody.

I know that, that we work with. A number of other people as well to support the organization. But I think that, that those will go up on the site soon. We want to also have like a community health team lead and we’re talking to someone about doing that. Job search is a big thing at virtual coffee.

It doesn’t matter what stage you are. Somebody is always looking for a job. And so we have some great folks who do a lot of work on that. And so, you know, that might be up on there soon too. So, you know, we’re, we’re, I think we’re in the phase where we’re trying to figure out how do we best support our members and help provide those leadership opportunities that we want.

[00:19:56] Michaela: Yeah. Yeah. So when you describe all [00:20:00] of that, it seems to me this is a full-time full-time job already, but so how much, how much time does really go into that? I mean, there’s the meetings themselves, right? That you’re a participating and I’m even, I’m not part of virtual coffee because I don’t have the time to do it as.

Just as a participant. So I can imagine you have to be at the meetings, you have to plan the meetings and there’s the chat and you’re making all this. You have all these thoughts and meetings also with other members of the community, how to grow the community, how to, you know, keep it alive and make it healthy.

So how much time off your week goes into data? I can imagine

[00:20:39] Bekah: a lot. I don’t know. That’s a good question. At some point, I think. Stopped keeping track of how much time was going into it because it was a lot and it’s not a job, right. It’s a volunteer position. But I think, you know, now we have so many supportive members and with the core team that I’m able to do, where like we’re all able to do a lot more and to lean on each other.

And to grow in that way. And a lot of the stuff is almost like muscle memory. Now, you know, I’ve been doing it for so long that it doesn’t feel like it’s one more thing to do. There are always things that, you know, I have a whole board of things I would love to do for virtual coffee and I have to try and pace myself because sometimes I go for it anyway, and then I.

Well into something and I’m like, oh, I, I might, I might die after that. So I try to avoid that feeling now.

[00:21:38] Michaela: Yeah, I can imagine. So I have seen on the GitHub page, there are some, there’s some sponsoring going on, right? Is there, are there other ways that you’re monetizing this community or that the community monetize it itself, that it has some budget around that you can also do cool stuff.

[00:21:55] Bekah: So right now, sponsorships, we launched [00:22:00] sponsorships maybe in September. And so up to that point, we were just paying out of pocket for everything. But sponsorships is the primary way that we cover our costs. We had a monthly challenge sponsorship, which was nice. We have the podcast there’s opportunities for sponsorship there.

Oh, oh, we just launched a store, so, oh yeah. Cool. That’s really exciting. It’s just really exciting to see people like wearing the virtual coffee and sharing their stickers. So those are some ways that we’re, we’re working on covering, covering the cost of what we want to do, and then, you know, hopefully providing new services and.

Yeah. Yeah.

[00:22:38] Michaela: I think at one point you have to think about it because even like this lag is probably not free, right. You have to pay per month membered and.

[00:22:46] Bekah: Nope. So we’re on the free version of slack because it costs, I think $6 per member per month. Yeah. I saw there’s no way that we could cover that.

That’s so

[00:22:59] Michaela: crazy. Yeah. Yeah. Okay. So there’s a free version of that as well, because I looked into that, I thought like maybe, you know, slack channel and it’s not like $6. Why am I God, not.

[00:23:11] Bekah: Right, right. Discord, discord, we’ve gone back and forth about it. It has a lot of great tools. We’re just not in love with the user experience of discord.

But you know, we have class. So one of the things that we have Put a lot of money into zoom because we have accounts for our core team, but also we have a coworking room that stays open all the time. So folks can join in slack and the coworking room is always open. So that’s its own account. We, you know, producing a podcast can be costly you know, producing.

Transcripts and providing the services for that. We’ve got like Zapier and air table. So, you know, like we’re, we’re using all of these tools that we can to you know, make things a little bit easier for us, but do cost money. And so what we, we try and keep our costs minimal, [00:24:00] but you know, there, there are some, but I think that.

Covered right now for our CA our monthly costs by our sponsorships, which is really, really great. Yeah. That’s

[00:24:10] Michaela: really good. Yeah, that’s really cool. So what would you say to the listeners today that would like to, you know, start their own community, have a community? What would be the MVP, a MVP version of a community that you, you know, from your experience would.

Suggest to them, should they start with fitness spaces or should they have like a meeting or a slack or a discord channel, you know, what are the options and what are the pros and cons for each

[00:24:38] Bekah: one of those? That’s really a great question. I think that, first of all, I feel strongly that. You, there are a lot of really great communities already out there, and a lot of them really need support.

So if you are not all on board and starting your own community, explore some of those and see how you can help because, you know, you might be able to be on a core team or something that allows you the experience that you want from that. So I’m not convinced that every, every person needs to start their own community.

But I would say that I think trying to fill a need within the community is a really great way to start one, because if you see that there’s a gap or that people are asking for things, or, you know, like one of the things we’ve been doing virtual coffee for almost two years now, and we get the same questions in our Our zoom sessions.

All the time. And so I can tell that there’s a real need for more work, to be done around interviewing about supporting junior developers about creating positive workspaces. So for sure there are. For groups that focus on those things. And then I would say for me, if you start a slack or a discord, that’s probably the most time-consuming [00:26:00] thing that you can do because you want to keep people engaged.

You want to keep them talking, you need to answer questions. So if you don’t have a core group of people, Then it’s going to be really, it’s going to be a lot of work to try and keep up with that. I also think that we’re in the pandemic now and people have been collecting slacks and discords, and when things in the pandemic start to ease up, we’ll see that a lot of those communities, I think, start to fade off just because, you know, people are going to prioritize the couple that they’ll keep and stay active with.

And, and then they’re going to be, you know, doing. In really stuff.

[00:26:42] Michaela: Yeah. Yeah. Which is good. I’m I’m waiting for that.

[00:26:47] Bekah: Yeah. Yeah. So I mean, thinking about like, okay, maybe you want to then create some kind of hybrid model or, you know, do an online meetup that translates into an in-person thing. Or if you, if in-person is not your thing, then, you know, figure out how you can build your online environment around that.

I think it’s tricky because it’s not one size fits all, but you know, in-person or async, if you’re a really async person, then, then slack or discord is a great way to go. So yeah, there’s, there’s a lot, there’s a lot there. Yeah.

[00:27:21] Michaela: I think it probably really about personality as well. I think a lot of people really enjoy.

Writing and, you know, participating in this estrone coroners conversation, even though they are often very synchronous right in discord. That’s why I always feel like I missed that conversation. Oh, I missed that conversation as well. And then I just leave without writing anything like anyway, so the last question that I have for you is you just started as a technical community, like.

What, what are you doing dead? Are you doing actually the same thing that you just learned yourself? And you’re not like your master now and the expert here for, for deep gram or what’s your role there?

[00:27:58] Bekah: Yeah, it’s kind of, I [00:28:00] feel like it’s such, it’s been such a good fit for me. My background, I spent 10 years teaching college English.

And so deep gram is a speech to text. AI company. And so there, there are so many different experts in different fields there. So, you know, whether it’s data science or linguistics or engineering and, you know, the devil team I get to talk to everybody and. Understand where they’re coming from, but I sit on the dev REL team as a technical community builder, so I can do dev rally things.

I can write if I want to I can contribute code, but my focus is on creating those systems and processes for community and the external community at deep gram. I always say that your community starts with the internal community. You want to make sure that you have a strong internal community before trying to start an external community, because you have to have that support network to help you and that trust to be in guidance.

So I’m doing. You know, some educating I am doing well, hopefully some speaking in the near future and hopefully some writing and building out that community strategy and trying to figure out, you know, where, how can we. Fill a need in the tech community or how can we support existing communities out there?

So it is, it’s pretty much a mixture of everything I’ve ever done in my life to this point. And it’s been really fun in the first three weeks now having a team to work with them. Yeah,

[00:29:31] Michaela: it sounds super exciting. Yeah. I can’t imagine everything coming together for you. And you can really strive now with the competencies that you, I think not only developed you probably had already from the beginning, right?

Because it’s not something that you. You make the first virtual coffee? I think a lot of people did that and then it grows into something that’s, you know, so probably not in the deaf community as well, so well rounded. So yeah, so [00:30:00] congratulations to that. And thank you so much for sharing so much about the process and about virtual copy, how it worked.

Yeah, I really enjoyed it. Is there something that you want to tell our listeners? Maybe how can they sign up for virtual coffee? He said, you know, do you, do you have to have some commitment there or accountability?

[00:30:21] Bekah: That’s a great question. So we make everybody come to at least one virtual coffee for before getting an invitation and to our slack and that’s to, you know, help them experience, you know, our community and to see what it’s like, because, you know, we feel that we demonstrate that pretty well in those meetings.

And so it’s really. Figuring out if it’s in the community for you, because it’s not the community for everyone, we all have different needs and, and things that we like. And don’t like, and so if it’s for you, then it’s great. Then join our slack, fill out our new member form. You can find those events@virtualcoffee.io slash events.

So come and check out a virtual coffee and then.

[00:31:05] Michaela: Yeah, cool. I will link everything in the show notes. And thank you so much for talking to me being here today with me. I enjoyed it.

[00:31:14] Bekah: Great. Thanks so much. I’m going to, can I mention one more thing? Yeah, sure. I just want to say to ’em if you follow deep gram devs on Twitter, I think we’ll be running some very cool Twitter spaces through there soon.

So if you want to check out some Twitter spaces, you can do that as well. And thank you so much for having me. This has been great. Yeah, I really

[00:31:35] Michaela: loved it. Okay. Thank you for caring. Thank you. Bye

[00:31:38] Bekah: bye.

Predictable profit through small bets

Daniel Vassallo left his cushy job at Amazon, where he made over half a million per year, to start his own business.

We talk about:

  • anxiety when start-up attempts do not work out as planned
  • how he overcame failure
  • his strategy of small bets to reduce uncertainty
  • and all the little products that provide him with an average of 23K USD of profit per month.
Picture of Daniel Vassallo

Today’s episode is sponsored by Codiga, a smart coding assistant and automated code review platform. Try Codiga for FREE!

Links:

Subscribe on iTunes, Spotify, Google, Deezer, or via RSS.

Transcript: Entrepreneurship as a developer

[If you want, you can help make the transcript better, and improve the podcast’s accessibility via Github. I’m happy to lend a hand to help you get started with pull requests, and open source work.]

[00:00:00] Michaela: Hello and welcome to the software engineering unlocked podcast. I’m your host, Dr. McKayla. And today I have the pleasure to talk to Daniel Vassallo, a former Amazon engineer, and now an entrepreneur and freespirit. But before I start, let me tell you about an amazing startup that is sponsoring the podcast Codiga.

Codiga, is a code analysis platform that automates the boring part of code reviews and let’s you merge with confidence on GitHub, GitLab, and Bitbucket. I’ve worked with Codiga for around one year now, and I love how it guides me in discovering and improving, well, the “not so nice parts” of my codebase. But there is more: Codiga also has a coding assistant that helps you write better code faster. Find and share safe and reusable blocks of code within your favorite IDE on demand while you are coding. Codiga has a great free plan. So there’s actually nothing that stops you from giving it a try! Learn more at Codiga.io. That is Codiga.io.

But now back to Daniel. Daniel got internet famous by sharing that he left his Kashi job at MSU. Yeah, half a million per year. Can you imagine? And his main motivation for leaving was that he wanted to be more independent, but also challenged again. After that you shared his attempts to build a successful business on Twitter.

And since then he managed to get a really large Twitter following. And even though his first SaaS business did not work out as planned, he ever reaches around $23,000 in profit each month. So I actually also asked Daniel if he would give away one of his products, to my listener and he agreed. So that’s really awesome. And so today you have the chance to win a couple of things, right? You can either win. Everybody can build a Twitter audience, which is a video course from Daniel, where he really goes into depth on how he, how he built this to the audience.

And how can you do the same? Or you can. When a digital [00:02:00] copy of the good parts of AWS. So it’s a book, a very technical book and tell us your, everything that you ever wanted to know about AWS, where he’s really an expert in. So how do you want to do that? Right. You ask them, you have to like, and retweet today’s episode’s link.

I will put that in the show notes and for an additional chance to win, you can also leave a car. On what are you doing to get a little bit more independent, some side income, or maybe a hobby that you have that just gives you more energy, which I think is equally important.

Yeah, but without further ado, let’s dig into Daniel’s interesting success story. Daniel. I’m really thrilled that you’re on the show. Thank you for being.

[00:02:41] Daniel: Of course. Thank you, McKayla, then. Nice to meet you. Happy to be here. Yeah,

[00:02:45] Michaela: I’m super, super excited.

So when I go to your website, I have a list of products that are linked there. Right. And they are very, very diverse, which is super exciting for me. I’m just two of them I already mentioned. Right. Which was a video course on how to build a Twitter audience.. Then you have the good parts of AWS.

Then you have cutting boards. They’re like wood, wood cutting boards. I was like, wow, how does that fit into that? Then you have like profit and loss, which is some status updates on your business finances. And then finally you have this SAS business user base which I’m also super interested in.

How does that come about, right. That you have like a cutting board in between. All of those is a little bit more technical parts or products that you have. Yeah,

[00:03:34] Daniel: yeah, yeah. So I think I sort of, I started this journey, I think in a traditional way, like many other software engineers, I thought I was going to be doing the boots.

I was going to sort of focus on one idea and build it inside to make it successful at all costs, putting all my effort into it. And I think what’s happened. And that’s what I started with. That’s what, that’s what user base. What is the light? When I, when I started [00:04:00] a few months in, I started to have a little bit of I was public call it like a small crisis of anxiety.

I was thinking I was getting very sort of uncomfortable by the uncertainty that I was imagining. In front of me, you know what, when we’re like, when will, I know if this will succeed? What if it won’t succeed? Like what sickness should I be watching for? What if it succeeds, but slowly that should I be shifting attention to other things and sort of, I had all these questions that I didn’t really have an answer for.

And I was noticing that I probably was underestimating. How much uncertainty that is in doing something like this, that it’s very hard to know. And th this was a bit more confounding because actually I was getting very strong sickness about user base back then, very song. You know, I just liked the term validation because I think sort of it implies something that it isn’t there, but the typical size that people look for.

For validation. Like I was sort of had a mailing list of about 4,000 people waiting for the launch. I had endorsements by highly influential people, sort of the CEOs of like Netlify and various publicly. And those, the products that I sort of and were very welcoming and sort of wanted, wanted to collaborate with their teams.

So many other tanks, even, even on launch. I sort of, it was front page on hacker news, number one on product hands, even, even the initial day 11, you were strong was actually about 40 customers, about $1,500 in sales. But yet I was still noticing that, you know, it was uncertain and I tend to find that that’s what happened.

It was much harder to keep the momentum of all the sickness that, that had been. So, so th that’s crisis a little bit opened my eyes. And I basically, I started thinking if I want to make this as an arrangement, the self-employment engagement, successful public, and it’s a different strategy. I can’t just be going all in into one thing, because I was worried that I was just going to run out of time savings and I have to go back to full-time employment, which isn’t the end of the world.

Like, it’s not like it would be the end [00:06:00] of the world. I was enjoying being self-employed. I enjoy the flexibility, the, the, the sort of freedom, the ability to work on what I wanted and all those good. So I don’t really know what has happened, but sort of I had the smaller epiphany and I wondered what if instead of having plan a plan B plan C and plan the about, you know, if this fails, I go to the next thing and so and so forth.

What if I decide to do a bunch of things at the same time, small things? Of course I was there. The stick. You know, I had limited finite time, like for the one. And that’s sort of changed my attitude to things instead of sort of focusing on one thing. I started looking around with me and I started asking myself, what can I do?

Low-hanging fruit. That’s what I kind of do. That is a small thing. Doesn’t necessarily need to have large upsides accounting because I’ve been new or, and this is, they be sustainable for a long time. It was mostly looking at small wins. That’s what, what can I do? Something that can be, can make me some money into the month’s time, which are these, the higher odds it’s still unpredictable.

You never really know whether something’s going to work or not. Even today. That was when I launched new things. I have no clue sometimes whether things are going to work or not. But it’s changed my perspective because before. Looking at things, does this have upside? Is this going to scale? Is this sustainable?

And I think that side of that comes with dads. Those things that do have those properties tend to have low odds of succeeding and they just to be hard, you know unpredictable sort of affected by them, them things, good timing and other things that are very opaque. And this has worked out well for me.

So now sort of the, to get back to your question, so to these and why now it looks like I have a very diverse portfolio of products is, is because I, I I’ve been the products are a manifestation of inspiration that I had over the last two and a half years or so. And I ask myself, is this something that’s going to be a small bag?

This is something that I can give it a shot. And maybe if it’s works, I keep it. If it doesn’t or if it becomes, you know, too, too, too, too hard [00:08:00] to do or something that I’m disliking, I can just drop it. So don’t, it’s been sort of a stream of experiments. Some things work, some things worked more than others, some things beyond my expectations, some things work, it worked a little bit, but didn’t meet my expectations.

So as you can imagine in a poetry. It’s like an investment. I mean, I did, I don’t think the idea is that we tend to do the same thing with, with how to, how we invest our money. For example, that we almost considered food. To deploy or your life savings, like in a single stock or a single assets that we tend to want to diversify, to tame uncertainty.

And I think more or less, it’s the same episode to my own ideas and my own small ventures, essentially. So I know it’s a general answer. I hope happy to dive deep or deeper into all the individual tanks, but sort of the general idea at the Titanic.

[00:08:52] Michaela: I think when I look at your projects and your products I think what’s also distinctive here is that they have started that end.

So a lot of people have a lot of ideas but there continues, right. So they are, I’m starting with it and that can actually not stop them. But it’s more something that I’m doing and. I keep on getting busier and busier because I’m adding things to my portfolio, right?

Let’s say I’m a SAS business user basis. Probably the exception here, right? It’s not something that you start and then it already ended. It has some maintenance cycle, as we all know, it’s software. Right. And if it gets more successful, you will have more, have to be more involved in. But like a cutting board has a start and end.

Right. You have the idea, you design it and then it’s finished and then maybe you have to sell it. Yeah. Okay. But there is not really a lot of involvement in it. The same for the video course. . You recorded it and now you’re selling it the same for the book. Right. You start it, you finish it.

so do you think that this is you, you deliberately think about that when you’re looking at your small bats? I mean, there’s [00:10:00] like as, as we have time, Infinitive ideas on board we can do. . And, and I think long enough people have struggled with that. I’m struggling with that. What should we, you know, what should we take on?

And there’s always a pros and cons list, or however people make the judgment is this affinitive thing of I’m starting this and, you know, in. Three weeks or in three months I have it finished and then I’m done with it. And so on. Is that something that crosses your mind that you’re thinking about it?

Okay. Yeah.

[00:10:30] Daniel: Yeah, yeah, definitely. It’s sort of, I think I’ve developed a selection criteria. I like to call it. It’s it’s sort of it’s I think, I think back when I started, I almost fell into the trap of. London can do the first opportunity to tie fell that I could do. And I would just jump into it. So I don’t, because it seems like, okay, I can do it.

Let’s do it nowadays. I’ve become much more vigorous. I would say that’s what my selection criteria and I feeling completely okay at peace ignoring things that I feel like I can do. I can give a shot, but if it, if they don’t set up. My strict selection. Cause I too, yeah. I just leave them for someone else or whatever that they’re just not mine.

And what you mentioned is a, is an important part of it’s that this is like, Is is this something that’s I like to call it? Is this something where time is going to be my friend, not my enemy. I think what you, what you’re doing is you, you need to have the payoff happened before a certain period of time. Otherwise you’re going to run out of.

Money or out of time out of frustration or get the motivated. And I disliked that situation. I disliked that property. So I try to avoid those as much as possible.

I think with user base, I also made the mistake of what the kind of SAS was. It’s the kind of SAS that’s acquired. Unfortunately in our 24 7 support, I have a PagerDuty app. I sort of, sometimes I get paged until 2:00 AM in the morning. Like nowadays I would almost certainly avoid something like that. So I thought I would go for something simpler almost certainly.

Other parts of my selection [00:12:00] criteria. I also is this something that I can build in blink to markets by myself? Not because I think I had no after the thing go, I can do everything, but I think it’s a good test to keep things simple. Right. It’s sort of a proxy for is the simple enough that I can bring good to markets by my.

Like if I change my mind I, or I decide it’s not worth pursuing, getting a more, I don’t have to make sure that everyone is on board or I’m not sort of haircutting anyone else.

Concerns go away. If I keep things simple that I can sort of do do, do on my own. . I just, with user base, I type invested so much that then I started falling into the trap of sunk costs.

Now I should invest even more. And I ended up spending, you know, a ton of time, money, and more than what my, this capital to the other was light. And definitely a lesson learned. And, and what you mentioned, I do like projects and ideas that sort of ideally have a start and an end where time is my friend.

[00:12:55] Michaela: So what are some of the products that you started or the ideas that you started that haven’t yet? That haven’t succeeded yet, or maybe that you haven’t even shared with the artists, some that you haven’t shared even with, with, you

[00:13:07] Daniel: know, with the people.

Yeah, yeah, yeah, absolutely. So there’s various defense of failure degrees afraid as well. Certainly I think we’ve already mentioned the user base, probably the biggest failure that I had. And this one I did it before. I was thinking about sort of the strategy that, so I think it’s a failure that led me to see things differently, but I almost feel embarrassed to say this, but I’ve spent almost $150,000 of my own.

Building user base. It’s crazy. It’s talking about what does, and this is only making about $5,000 in annual revenue, not monthly. That’s almost certainly I will never recoup my money may be I might sell it in the future, but even though I’m not expecting a sort of large amounts, and I don’t know if even that is an option, but as I mentioned before, I think my mistake was in the.

Underestimating uncertainty. I invested too much. I went to all, then the more I went all into my life as I can need to invest. And it was sort of this [00:14:00] cycle of, of bad decisions, but think that has side. However, I do a few other small things that I would spend, not necessarily they weren’t big failures, but there were things that didn’t see the work as I expected.

In fact, the very first digital product, the tight side, not many people know about. Was I’d say to sell a spreadsheet there’s a bit of a technical project that I started to sell a spreadsheet about benchmarking different, easy to servers during network speeds. So, so back then, back in 2019, AWS didn’t really advertise accurately.

What network speed you are getting and what network bathymetry are you getting when you’re buying or using a particular instance type? I thought I’m going to run a benchmark test and measure exactly what type of performance you’re going to get.

I was building in public and talking about, as people got interested and tights, basically long story short, I spent a week cause I was. Open source the tool had benchmarked. Everything spent about $500 I did prepare a very sophisticated spreadsheet with pivot tables. Very fancy spreadsheet. I was very proud of it. I posted it on Twitter. Post is on hacker news and a few other places. There was lots of fences into this and that tension that’s actually surprised me.

I think the sort of lending page got about 30,000 views, which was even nowadays that’s compared to my other products, but this sold nothing. Cause I mean, nobody paid and I was, I was charging $10, like not like a crazy amount of money, nobody bought it. So I tend to, in fact, a few days later I released it for free.

I, and I think. Lesson that I learned that is back then. I remember thinking, wait a minute, this is what back then it was 2019. I, I said 2019, you know, information wants to be free. Nobody wants to pay for information, especially developers, but I think actually I’m happy that I didn’t sort of take that lesson too seriously because it was, I jumped to the wrong conclusion.

And my now in hindsight, it’s not really to do in fact. Money now selling information, different kinds of information that’s and sometimes even to developers, that’s sort of that belief or that [00:16:00] conclusion, the developers don’t pay for things. So in fact, I think a lesson that I learned with sort of these experiments is that sometimes we tend to jump to conclusions. The tire not warranted.

So that’s that’s that’s why did this fail? Why did, why were people not willing to pay for it? Maybe I can take some educated guesses,

but nowadays I’d say to be careful not to be too much from these things, because I know that things that are affected by that. bye. Good timing and other unpredictable things that sometimes whether something succeeds or not is just things that we can’t even see that it’s just opaque. Yeah. So that was another example.

You know, we mentioned the cutting boards, so the cutting boards started as purely as a, as an experiment. I started with working just this time, last year as a hobby, . And at some point I was wondering, could I try to sell some of the things that I’m making? And.

Oh, eventually I bumped into the idea of within cutting boards I launched the sort of the offering, I think in October of last year. So maybe four months ago, I sold about $5,000, not a big tank, sort of probably if I, if I consider all the tools and the tank, I might have just broken even .

So it’s sort of it’s that failure in terms of expectation. Nevertheless, I still think that I gained something from the novel. Because I did ship F you know, I bought a dozen around the world and learned a lot. But I think what’s more important is that now, nowadays I feel much more, I feel less daunted by a physical product offering.

So you know, the, the, you mentioned, for example, I have the profit and loss membership.

That’s another thing that was purely an experimental. If I was so a bit of, a bit of a background story, I always also have sort of this quarter time freelancing arrangement with. Are you familiar with gumbo such as this platform for, for, for digital? And back then when I, when I joined the team we were launching a feature memberships that the ability to, to, to do the CareLink sort of payments.

And I wanted the next cues to try to use the feature. Essentially, this was, this is how it started. I [00:18:00] was wondering what could I sell for the carrying fee? And I had already been very, very open with my finances for a long time on Twitter and other places with my business finance, at least like with how things are doing.

And I talked, what if I talked to. Sell an even more detailed financial view. Like we built and done by products, broken down with all the expenses and where the costs basically the full profit and loss per product. And I started at then when I started it, I was hoping that I could build like a community at ons.

And then with these ideas of making money online and figuring out small bats and building a portfolio. I could use the circle group. I was posting updates. So not only I was posting my financial desires, but some comments as well, long story short, even though this financially know did okay. I did about $40,000 in total in about a year or so.

I’m quite happy compared to the input I’ve put, but sort of the, the, the, my goal of building a community, it didn’t tell the happen, like. Originally, there was some momentum. I was getting lots of questions, but it never really changed some of my talking about my own topic and eventually things sort of Sort of stops the energy disappeared and barely any activity continued to happen.

Nevertheless, almost by surprise. These simply my mostly synthetic jackets actually is a cohort based cars where we’re meeting 25 members at a time that I launched this about two months ago or so where we were having sort of this synchronicity. Basically cars that we meet together, a small group. And we talk about these ideas in much more detail that we’re discussing here, sorts of finding inspiration, finding opportunities, juggling multiple things at the same time.

And funny enough, even though it wasn’t my intention, I set up a discord server just for housekeeping purposes to share the zoom links and the slides and whatever, and the community that I was hoping to build back then sort of happens there now by accident that this wasn’t something I was. The time to do it, which is fascinating.

It sort of became the [00:20:00] community almost became the main selling point of this course that people are word of mouth is spreading. You know, there’s about 200 people now, like, and people are encouraging others, you know, come here, we’re talking and getting good feedback and brainstorming ideas and things like that, which is again, it’s quite fascinating to me.

Most of the things that have been successful and even the failed things, but particularly the successful things are things that I have wouldn’t have even imagined I’ll be doing. Just a year ago or so that’s what sort of ready for the interesting sort of the, I had the idea before I used to believe, you know, business success is going to be a, a self directed top-down you come up with the idea and you execute it.

And now it’s almost, I tell myself that, send them things, see what works. So I wait to that. Yeah.

[00:20:48] Michaela: Yeah. And it’s also interesting to see that while you have this idea of having a community and then you have an intention and the deliberate effort to do it and it doesn’t work out. And then it’s. By chance somehow creates itself.

Right? I think this is really an interesting perspective. I want to dig a little bit into your motivation and, do you think that you will be bored by what you’re doing right now

and what is really your motivation behind that? For me, for example, I’m definitely an entrepreneur because. I want to have the freedom to be with my kids. And I didn’t see how I can do that in a job that fulfills me being employed just wasn’t possible. There was nobody that would offer me the flexibility that I need, right.

Working in the middle of the night or on the weekends, or, you know, having like crazy two weeks of work and then three weeks doing nothing because the kids are sick or, you know, and I feel like this is more important now for. And that’s definitely my motivation for being, I think self-employed, what’s your motivation.

Why do you, why do you do that?

[00:21:52] Daniel: I think the motivation to become self-employed I think is similar to yours and similar situation, I would say. So also two small kids[00:22:00] I was feeling like I’m leaving the house before they wake up.

I live home, very tired. They’re about to go to sleep. And in the weekends, I also thinking about work. So maybe it may be a bit different for me as a dad, but still sort of the same concept that I’ve felt like, you know, my kids are going up and they spend time with them. So I was looking at people around me at Amazon and there was noticing, you know, things are probably going to get worse, not better.

I wasn’t envying the lifestyle and being, getting a good sense of other people, no matter how much more they’re getting paid or how much higher up they were. So I’d be fooling myself. If I were to believe it’s going to be different for me is if I just get the next promotion or the next bonus, suddenly things are going to be better.

So it’s I jumped into to the self employment. And then once I did.

Get all this flexibility I take immediately. I realized that I didn’t see the one, this to be taken away from me. So suddenly I w I joke sometimes I joke tweet about this as well as my own. The business plan is not to go back to a full-time job. I’m not thinking about goals that I want be. A successful SAS or this particular product or some idea or whatever, it’s literally to make this lifestyle sustainable.

So that’s, I would say is it’s, it’s a different, I think it’s a different attitude. And I think it’s helps me with my decision making process as well, because when I do bump into these opportunities, so I get inspired to do something. I ask myself, is this going to increase the odds or is this likely to increase the odds of making this lifestyle sustainable or not?

[00:23:28] Michaela: It’s true. Yeah. We started sort of at the same time with the entrepreneurship.

And I’m a big believer in that you have to enjoy the process, which I think is a little bit the different way of saying the same thing that you just said.

Right. So I only take on things where I feel it’s not the end product that I will enjoy, but I enjoyed the way. The only problem. And I tried to combat that, but I think it’s very deeply internally in my, in my, in my genes or something. DNA is That I have a hard [00:24:00] time having the money focused all the time.

Right? So that’s for example, say I did a PhD, right? And from a money perspective, this is such a stupid thing, right? It’s the worst thing. The worst decision that you can, that you can make is you’re not making, you know, money for a long time. You’re putting a lot of effort, a lot of work into something. But I think I wasn’t, you know, I wasn’t interested in the money.

It was okay for me doing it just for the process. And I survived in a small apartment, you know, like we were thinking of, you know, it should be by this this vegetable or not, it was definitely this kind of grinding. But then in hindsight, I really try to do things different than now because the sustainability part, you know, is also a very, very important part for motivation, right?

You can be extremely motivated and it can be such a fun to do something, but in the end of the day, and maybe this has to do with growing up and having a family and, you know, it’s not only you and you know, your crappy apartment, but you have to provide for your family. And, you know, and this comes with, with, with this aspect of also sustainability.

I try to have that mindset much more of thinking of is that actually something I can keep up doing. And unfortunately, you know, money is it’s a big part there, right? Is it, is it coming very naturally to you that you, that you think about those?

[00:25:22] Daniel: I, I actually, I think actually I there’s an, it sort of idealized that I am almost incapable of thinking properly about money, unless I feel I need that.

So that’s, and I think. I don’t know if it’s a different situation. Maybe that’s like now my wife is staying at home with the kids, sort of like now I’m I’m diploma income person. I tend to sort of, I think that stressor of needing to make ends meet soon because otherwise I would have had to go back to work.

I think that was probably the most critical thing for me to open my eyes and find [00:26:00] opportunities, ignore and not ignore. I would say that instead of being. Prudent and not being too idealistic and making the making better decisions essentially. And even now I noticed that, cause now, now with the types of work that I do, I have lots of ups sends hours.

My income is very volatile. If I launch something new, there’s like big spikes and then sort of zaps sort of very spike. So if I’m noticing there’s something very fascinating that I never noticed before. That’s when I’m riding the momentum of a, of a high period. My mind almost shuts down. Like I can’t work.

I can’t think I’m not creative. And I think this is similar to what creators talk about when they mentioned your constraints, bleed creativity. Now, for example, early 20, 21 last year, the first. I would say seven months was a good period for me.

I was riding the momentum of a couple of other projects that I, that I was doing. And I realized I don’t need to think about money. I’m just going to experiment with hobbies. We took a two month vacation that we were on back to you with it. I’m from originally. And we spent a whole summer there, like.

Just almost didn’t do any work until September, basically. Right. But then sort of by September things have dropped down to a point where it wasn’t critical, so there’s a buffer of savings or whatever. But I started to notice that if I’m not going to do something, things are going to start becoming negative soon.

So that was enough. But then suddenly I can almost see it. I can almost feel. I started thinking of new promotion ideas, new products, new opportunities, and very quickly I went to do September. Then I went to a very high focus mode. I started with working thing. I started the cohort based course as well for as much a month after I did a few new promotion campaigns for my existing product sense.

You know, again, I could change the subject to the, again, sort of, again, not everything worked, same, same thing. Cause somethings work [00:28:00] better than. But it’s sort of very fascinating, you know sort of it it’s it’s as if the assessors are what I need to make me make design decisions. I will feel almost incapable to work if it doesn’t.

And we talked before we started recording that, we talked to this a bit about the, the plot act of building a house custom built house. But I think it’s helping me as well because it’s giving me some, some meaning to my income, I think if I didn’t have debt, I felt that we were working much less a public, those Newlands would just sleep in my head and just, I would be less creative, less productive.

And I would be seeing the words differently and probably opportunities that. I would consider now I would ignore, I call it luck blindness. Like sometimes when you’re in the state you might bump into opportunities, but if you’re, if you don’t have that small successor, you almost don’t even see them.

The lights is just because we’re shut down. So I don’t know if it’s, I think I, I understand what you’re saying. Maybe the circumstances are a bit different. No.

[00:28:59] Michaela: Yeah, I can totally relate to what you’re saying, especially with the house now, because the house shifted my whole thinking around money tremendously.

I’m a very frugal person, extremely frugal. I have always been my whole life and you know, it’s just didn’t mean anything to me money, but you can’t be frugal with that. I was actually thinking like, now that we get our. We have a very, very small car. And there are no, there, there’s no way that you can fit three child seats into the last row. Right. There are just two and a half seats. So now we have to buy it.

Bigger car. And so I was brainstorming with my husband and it was like, okay. So, and, and it really helps me to think about products and my wig. It gives so much meaning to me to thinking about, okay, this quarter of the car can come from that. And, you know, I could do this to get the other. And so [00:30:00] having, having a very realistic way of how to spend the money definitely is extremely different.

Or life-changing for me, for my mindset, because, you know, having the managers on the bank account, or it has no meaning for me, but now it’s, you know, part of a car and can a kid sit there or not. Right. And so on. Yeah. And I think that I wanted to ask you and maybe it’s probably the final question that I have for you.

And that I think is super interesting for my listeners is how do we deal with scope creep? But also how do we, how can we let go? Because for example, one of the things, when, you know, when I looked at you and especially at the beginning, You were only on Twitter, right?

So you tried a little bit to be on medium, but very quickly realized, you know, you have a couple of very dedicated and deliberate clock posts. But you never started really like your blog or now, or you’re building like your audience on the blog. You were building your audience on Twitter. And this, I found that really inspirational because I was also like, I didn’t have enough time to be on Twitter and on Instagram and on, you know, like what not.

And then you’re doing a shitty job on all of them and you just but now when I looked at your website, I saw. Oh, you can find me on Twitter. Gumroad Instagram and clubhouse. And I thought like, is that a little bit of a scope creep? Is that are your you’re you’re now on many things, do you feel like you’re, you know, are, do you still have like a very deliberate way of choosing where you are and how to also say, okay, I tried clubhouse and maybe now it’s not working in a.

[00:31:37] Daniel: Yeah. Yeah. I wish I had the precise on. So I think it’s mostly goes by Gutfield to be honest, I think in fairness, this as a full disclosure in the beginning guide side, many, many different things, as you noticed, I was on medium, but not just that I was, I think, little bit of background story. I think what I realized the very first week when I started working for myself, I realized I [00:32:00] had no reputation.

Oh, outside of the couple of companies that I had worked from, nobody knew who I was and there was still, I wasn’t, I wasn’t thinking, building an audience back then. I was thinking, can I build some reputation? Can I make some people know about me? And I was thinking, how can I go out there and help?

I was doing some open source on that data was very active for a few months. StackOverflow LinkedIn Quora, Twitter on medium on hacker news, LinkedIn, almost everything that had given that side. I dedicated probably a couple of months just experimenting different things. Some things I gave up quickly.

Some things, it took me a few more months, long story short, mostly Twitter was the only one that stuck around. And if you were to ask me what were the exit signals that made me choose that? I constantly say exactly. It just felt there was something that was working better. The return on investment seems to be better.

And again, almost like procrastination or something similar, a cousin of procrastination, something was making me feel like what’s the point of going off of spending more time on stack overflow. If this is not conducive to more people knowing about me and that sort of thing. It’s not, it’s not that I wasn’t helping people there, but it wasn’t a translating to me building a reputation, which I know sounds a bit selfish.

But in that period, that was my goal that I wanted to get myself no, a little bit. And over time I still experiment. So I sort of cocktail party downside. But again, for some of these and I, I lost motivation, right. I, I stopped. I stopped going. But so I still keep my eyes open for various reasons, because again, the opportunities diversification as well, sort of who knows I’m nowadays with Twitter, I feel like I have something to lose.

This is a common problem.

Since I have, you know, a hundred thousand followers or so. Concerned, you know, to some degree that’s what if that were to disappear? What if the algorithm changes? What if I get kicked out for some of these who knows or [00:34:00] get hacked or whatever? So again, there’s the back of my mind, ways of diversify flying that I haven’t really found anything concrete yet, so I, you know, it’s not scope creep. I was called it, I think is just still part of my experimentation phase.

Yeah.

[00:34:17] Michaela: I think that’s perfectly fine. I think it’s really important that we experimented with try out thing. And what I like what I also do is that if I feel it’s not like, for example, Twitter, right? That could be that I’m not posting something for a couple of months just because I don’t feel like it.

And, and I really liked this freedom and maybe, you know, like there is no comparison between, you know, your Twitter account and micros with your account. It’s really a small account by. I don’t want to be like the slave of Twitter, right. Or whatever I want to have, you know, like, and sometimes like in the summer or something, and there was a lot of stressful time or even a beautiful time.

Right. Sometimes I just didn’t feel like I have something to contribute and, and then I’m not, I’m not doing it.

[00:35:07] Daniel: Yeah.

Yeah. And, and, and, you know, I’m, I, I’m really not a fan of sort of the, the, these ideas that you need to be prolific and tweeting all the time, all the days. Sure. That might be the best strategy to put the audience faster, but what’s, what’s the point as you were saying. I think what I wanted to my, my, my ideal situation is that this becomes something that I enjoy doing because.

If not chances are it won’t last. I told like I, in summer, I think between June and August, I almost didn’t tweet at all because I was spending time with my family. You know, I might have tweeted it a couple of times. Yes. You know, my follow workouts stopped and maybe even lost some, but like, who cares? So I did sort of.

This idea that everything needs to be done in the most optimal way, sort of the fastest coat subject to the, whatever. It’s just very, as you mentioned, it’s sort of it’s you become a slave of, of, of the process, but in [00:36:00] general, in general ideas, like these are all extrinsic, things like that. I think they feel very naturally.

That’s sort of, you need to come to keep the streak uninterrupted streak of posting daily or blogging after a week or whatever. It’s things that I feel. Those things that I feel liberated once I break them, that’s a sign that I should probably not be doing them.

[00:36:20] Michaela: Yeah, exactly. Yeah. You know, this is why I’m, you know, an entrepreneur to have this flexibility and freedom to choose. So, but I don’t want to take too much of your time anymore. Then you thank you so much for being on my show.

I really enjoy talking to you. We could, you know, continue for another hour, but let’s be very deliberate with our time as well. Just want to remind my listeners of of the chance to win, you know, one of your products, either the good parts of AWS, the book, or the how to build a Twitter audience video.

So retweet, like the episode, maybe tell us what you are doing, that you enjoy, that you know, is one of your small bets to have an additional chance. And yeah. And that’s it. Thank you so much, Danielle, for, for being one of my show was

[00:37:07] Daniel: really a pleasure. This was very fun ticket. Thanks a lot.

[00:37:10] Michaela: Yeah, it was great.

Thank you.

 

Make money with open source software

Alvaro Trigo is a web developer who could quit his full-time job due to his popular open source software FullPage.js.

We talk about:

  • how to use open source to make a living
  • how long it took him to build software people want to buy
  • what he does against fraud
  • and his advice for developers that also want to go independent with open source software. 
Picture of podcast guest Alvaro Trigo

Today’s episode is sponsored by Codiga, a smart coding assistant and automated code review platform. Try Codiga for FREE!

Subscribe on iTunes, Spotify, Google, Deezer, or via RSS.

Transcript: How to make money with open source

[If you want, you can help make the transcript better, and improve the podcast’s accessibility via Github. I’m happy to lend a hand to help you get started with pull requests, and open source work.]

[00:00:00] Michaela: Hello, and welcome to the software engineering unlocked podcast. I’m your host, Dr. McKayla. And today I have the pleasure to talk to Alvaro Trigo, a web developer, who enjoys learning every day. But before I start, let me tell you about an amazing startup that is sponsoring today’s podcast. Codiga. Codiga is a code analysis platform that automates the boring parts of code reviews and lets you merge with confidence on GitHub, GitLab and Bitbucket.

I’ve worked at Codiga for around one year now and I love how it guides me in discovering and improving, aell, the not so nice parts of my codebase. But there is more. Codiga also has a coding assistant that helps you write better code faster. Find and share safe and reusable blocks of code within your favorite IDE on demand while you’re coding.

Codiga has a great free plan. So there’s nothing that stops you from giving it a try. Learn more at Codiga.io. That is Contiga.io.

But now really back to Alvaro. Alvaro created his first website at the age of 16 and did not stop learning and perfecting his craft. One day, he started to make JavaScript plugins and well again, to learn and as an experiment.

And three years later, he earned enough from his experiment so that he could quit his job. When I last looked his library. 21,000, I think downloads per week on NPM. And today I’m super, super excited to talk with him and learn how he actually makes money with open source, how he could, you know, write something on the side so that he can.

Quit his day job and become free and independent. So hello, Alvarado to the show. Welcome. And I’m really excited that you’re

[00:01:46] Alvaro: here. Hello, Mikayla. I’m happy to be here.

[00:01:49] Michaela: Yeah. So the last time I looked and this was 2020, right? The feed from 2020, it said like 21,000 downloads on NP. M what is it [00:02:00] now? Like? It must be even higher.

[00:02:02] Alvaro: Well, to be honest, I’m not sure I haven’t taken awhile, so I don’t know exactly, but, uh, so the similar, I guess I’m not very sure if the MBM stats are very representative anyways, but I think it’s just a way to, to show that, you know, you can have quite weight broadcasts. If you do something, you know, by yourself in your own house, nowadays is quite easy to reach so many.

And to, you know, to influence people in some way, right. Or companies or whatever. So I think it shows the power of open source in time.

[00:02:30] Michaela: Yeah. Yeah. So this is exactly what I want to deep dive a little bit into the power of open source. Right? I think the power of remote work. Yeah. It’s something that I’m very fascinated with and with the pandemic, I think a lot of people now really understood what’s going on here, but what about open source?

How did you even come up with the idea and maybe we should also explain, well, listen as a little bit what you were building, it’s called food page GS. Right? And as I understood it, it’s like a webpage that goes over our website that goes over the full page and it’s easy to set up. And,

[00:03:03] Alvaro: um, it’s a component that allows developers to create these kinds of websites, where you have like a full screen kind of slide.

That snaps to each of the sections. So when I created that, the fake wasn’t very popular yet it became very popular because I realized that almost at the same time that apple released the page four, iPhone five C the iPhone five C website was using these kinds of effects. So I think then people started searching for these kinds of effects and, and they were able to find my.

So, how did they come up with the idea? Well, I, I didn’t actually think too much about it. I wanted to create a component just to learn by myself. I wanted to keep on practicing jQuery at the time because the first version was a jQuery plugin. It wasn’t vanilla JavaScript. And well, I just came up with this idea that I thought it was cool because I was making a website in my.

At the time my boss told me, I, won’t kind of like a very simple website, kind of like a PowerPoint presentation. [00:04:00] And then I, I restart it to me. Then I came up with this idea that, uh, I took from different websites. And then after grading the website, I thought, well, this was quite difficult for me. I was trying to find for a component to do these kinds of websites and it doesn’t exist.

So it might be a good idea for me to create the components and see if other people find it also. He goes, he was out of that, saved them lots of time. And that’s what I did. And then decided to get some traction on GitHub. I remember one day. And seeing that you had like 500 more stars on GitHub and I didn’t even push it too much.

So I was quite surprised and then, uh, just kept moving. Cool.

[00:04:35] Michaela: And so did you, ’cause I, I looked on, on your GitHub profile in the size now, GDPs and GPL, right. Was that the initial license also that you looked up and you thought this is the right thing. Did you already think that you want to commercialize it a little bit?

Did you want to make money by. Having people that use it commercially?

[00:04:57] Alvaro: No, not at all. My whole purpose was just to, to keep on learning and at the same time to get the motivation for myself to, to, to keep on learning, to do something useful for others and that, you know, well, it makes you very excited about that and it needs to lead.

It was under the MIT. Because that’s what most open source kind of use writing in JavaScript the spectrum. And they they’re all, when I decided to start commercializing it is when I made the change to GPL person three, because I thought it was going to be more suitable.

[00:05:26] Michaela: Yeah. It’s also my license, like know it’s the Apache like years before it was the unpatched.

And now it’s always the MIT that I, that I use for open source. A lot of freedom to the user. Right. And so does it mean that if you already licensed it before under MIT that’s the previous versions are still licensed under that and only then the new things are under, you know, GDPR or how does that work?

I’m not really

[00:05:51] Alvaro: familiar with it. I don’t see this thing is quite complicated, so we understand I’m not really. Uh, an expert on the topic, but yeah, I guess what happens is [00:06:00] so full page version one and verse two, and they’re the MIT license you’re going to still find them under their releases sites and you’d have, and then after that I changed to GPL.

I don’t know how that legally works or I don’t know exactly how it does, but.

[00:06:16] Michaela: Yeah. Yeah. I probably, it will work like this, at least. That’s what I thought, but I’m also not, I’m not alone. I have no

[00:06:22] Alvaro: idea about it, but,

[00:06:25] Michaela: and so, so how long, because there’s also another tweet of yours and you were saying, well, I started to play around with that experiment with JavaScript, plugins and so on.

And now three years later, I actually quit my full-time job. How long did that really go? And where the times where you thought. This could be something or, you know, a lot of a long time where you just did it for fun, but never imagined that it’s, you know, going to be your bread winning. Um,

[00:06:56] Alvaro: yeah. Well, at first I couldn’t even imagine that I was like, somebody would be able to leave out from, from these kinds of things.

So I guess it’s kind of not right to me. I was working on it for free for three years under the MIT license. You know, at some point you start getting a bit, you have to take the seasons like, so that I keep on improving this or. And go for a barbecue with my friends. Right. Or, uh, so that has been extra hours doing this after work or so that just chill with my housemates or my friends or whatever.

So, you know, at the beginning of it’s very exciting. You, you get people using it and that excites you a lot. And then, um, you know, provides you some, um, help to keep on improving it. At some point you start having to work on it when you don’t actually feel like working on it. Right. Because people keep.

Requesting new features or they have bags, very specific bags that almost look like you are doing free consultants services for them. Right. And that’s when I thought, well, you know, maybe I can try to get something out of it. I got motivated by another developer it’s called. I don’t know how to pronounce it either.

I think it’s the Sandra, he created clickety Mansuri and [00:08:00] another popular JavaScript components. He was providing them the licenses for them paid like. I went into those, I read the bit how he was doing, and then I thought that I could do it as well. And that’s when I started charging for not for the product itself, but for paid extensions to the plugin.

So they were non opensource life extensions that I was selling on top of the open source project. So that’s how it just started. And then I saw people start buying them and that’s what motivated me to keep going. And then I noticed that I was more, I was feeling better about it all. Like if somebody reported the bag or wants it, this new feature or whatever, then I wouldn’t be so bothered about it because now I knew I was getting something in return.

So I think this way it works a bit better. Like when you can take the time to do something and you get compensated for that. And then you don’t feel as bad because you can, you’re not just wasting your free time. You’re actually doing something in your work time, I guess. And that’s a win-win for both sides, for the developer and for the users who get better support, better features, you know, better response times.

So I started sending the licenses, sorry that, um, extensions. And then after six months of doing that, I was able to quit. And then after a year or so is when I decided to also change the whole project and our license and try, uh, charting for the, for the license itself for commercial use, for non open source projects.

Yeah.

[00:09:33] Michaela: I think that it’s really, really important that, you know, projects, you know, that they are sustainable, especially if you have some success with it, right. Because. There is more and more time that you’re spending on it. And I think that, you know, nobody, like if it’s a hobby and you’re spending one or two hours per week or something, this, this idea doesn’t come up.

Right. But if it gets, as you said, if you have more users, you’re getting more requests, you’re getting more ideas maybe for features. So you’re [00:10:00] spending more and more time. And obviously, I mean, we all have to live somehow. Right. Um, obviously you have to think a little bit about how can I actually. You know, make that maybe my, my job.

And I think, especially if you like something, that would be like a win-win. So yeah, I can totally understand that. Another question that I had for you when you were, when you were turning the drought, I saw you have really big names. You have big names on your website for companies that are using full-page.

Yes, right. There was like a Google eBay, Coca Cola, Sony, and so on. Right. Did they all purchase a commercial license? Um, you know, did you reach out to them when you changed your licensing model and your, you know, your monitorization idea around that? Did you reach out to them and say, you know, I’m actually by now licensed for that?

[00:10:48] Alvaro: Or so some of them are using my license. That’s how I managed to discover, uh, website. Cause when you use the license for, when you buy an extension, you have to register it for a specific. Extensions of a different price, depending on how many domains you wanted, he wants to use extensive for those. That’s how I use for some of them all there’s I think I just discovered them by chance and they might not be using the license.

So it depends, but no, I, I didn’t read when somebody buys an extension or a license for the product. Now I get there. Um, I get to notify them about dates or changes. So yeah, when I released a new version and notified all of my previous customers and told them about that, that if they were not previous customers, they didn’t get a notification.

If they were using the previous restaurant, they can keep on using that forever and nothing’s going to change in their sites.

[00:11:37] Michaela: And so what’s preventing people from using. The new version and not buying for a license?

[00:11:45] Alvaro: Well, I guess some people, some people might not know that there’s a more recent version and others, I guess they they’re happy with their website and they don’t want to touch it at all.

So they don’t even want to bother to update to the latest version. All there is, they might not want [00:12:00] to pay. It’s not very expensive, it’s quite cheap, but you know, there’s all kinds of users, not only companies, but also freelancers or people who just make websites for. So that might be one reason because there’s no other good reason to not update because it’s totally compatible.

You don’t have to almost change anything in your code. I think it’s just one line. If, so

[00:12:19] Michaela: the question was more, um, are there people that are updating. Not paying if they use it commercially or is that not possible? Do you have some prevention mechanisms around that? What I mean is that, can I, you know, can I be a little bit cheat, cheat your sister?

[00:12:36] Alvaro: Yeah, you totally can. And there’s not much that we can do about that. I mean, yeah. For example, I discovered there was a McDonald’s website in Russia making use of day of the new version of the blogging with a new license and they were not paying for it. And there was a warning if you opened the JavaScript.

You were able to see a red warning say, Hey, is this somebody is not licensed. You can get the license here. There’s there’s not much that I can do about that. Uh, sometimes you can try to send, I think they’re called DMC a notices to Google or to some marketplaces to take down some of the staff at, you know,

Yeah.

[00:13:13] Michaela: Yeah. So you are more focused on, um, directing your energy to creating more value for your users and, you know, doing what you like and not so much buttered with, with the thefts and, uh, you know, trickeries and things like.

[00:13:27] Alvaro: I I’m only a little bit, but I try not to bother too much. It is true that with extensions, with, uh, extensions to that, to the main components in those, you have to register each of them for four different domains.

So you actually have more like, um, yeah, there’s kind of like a more sophisticated license key that you cannot trick so easy that that’s only on next sentence. I don’t want to enforce that on the product because then it will be too much of burden for many people. Yeah.

[00:13:57] Michaela: Yeah. And I mean, I think I would [00:14:00] probably draw a line a little bit between, as you said, smaller, you know, maybe individuals that even though I’m in $10, come on, but you know, like if then they’re like larger organizations like Google or E-bay.

In addition to this small fee, really small fee for you, are they providing some support? Do you get some sponsorship like there’s GitHub sponsors or something, or do you have other ways to, to support your work?

[00:14:26] Alvaro: Well, what I do is I also know I don’t, I don’t do sponsors and they don’t usually get support unless they go for a higher, higher license.

They do get support with. All kinds of tires get support for those, but not for the main license. And also what I do to get money from different places is also, uh, I sell, uh, against. I know you also do some affiliate things. And I also have another flagging that I also commercialized. So I have a few things that also drive money and not only the Fulbright silences also like sentinels, which are not opensource.

Yeah. And, uh, well, uh, I just remember that another way that, um, many people don’t even bother to get a license is because they don’t know very well how the GPL license works or, uh, you know, they are not very after with license. Terms and stuff. So they don’t even know that they have to purchase the license or they didn’t even know what is considered opensource or how to add a license to their own private.

So, this is a big issue that I see, especially on smaller developers, freelancers.

[00:15:31] Michaela: Um, so you were talking about plugins. Is it like a package or is it like a plugging? Was it the difference here?

[00:15:37] Alvaro: So well, back in the day is every time you made a component for jQuery, it was called a plugin plugin for jQuery.

And nowadays I tend to call it the library and JavaScript library, but yeah, also, um, the, I talk about plugins for work. Because a WordPress, you know, they have these kind of flagging stuff and then it’s flagging that it incorporates the library and allows you to [00:16:00] do certain things and you have a little interface.

So it’s not just for something on top of the company. That allows you to do things through an interface. Yeah. So

[00:16:12] Michaela: w when you were right at the beginning, you said, well, I tried to do that for my, you know, for my boss and try to implement that. And I was surprised how difficult it actually is. Right. I’m, I’m really in the same situation because I ported two websites from.

To get SPI now. Right. And I was like, oh, I studied excited, uh, you know, generator and gets me super supported and so on. And I completely underestimated how long, right. Even though there are so many plugins or, you know, uh, you know, components that you can use. But just even researching which plane to use, then writing the query really adopting it to your thing.

I mean, it’s like, it really blew out of hand. And even though their websites are, you know, now almost done, it’s like always like, oh, this little bit, you know, still like that, you even have to think about. I’m like, yeah, for work, like with word press, I was not so happy because it’s slow and I didn’t want to be bothered with, you know, going around PHP and, you know, changing something in the back end to improve my speed.

And then the loading time off each page really was annoying. So I thought like, okay, I’m going to do a guest beside, and then I will have everything with markdown fires and, you know, directly gets them from the fire system and it took. Actually no time almost right. To have that set up because there are already like start-up packages and then, you know, like, but then the small things like that, the canonical link is,

[00:17:40] Alvaro: you know,

[00:17:42] Michaela: it’s horrible.

Yeah. That I have a site map and then I have the keywords and I completely underestimated that. So, yeah, I totally understand. So if I want to use full page now in my guests beside, is that, is that something. Is that a thing? Can I do that?

[00:17:59] Alvaro: Would that [00:18:00] make sense? Yeah, I guess, I mean, well, I don’t know what your site looks like.

Is it more like a

[00:18:05] Michaela: blog? Um, it’s mainly a block, but it’s not only a blog. It also has like a run pages, right? Where landing pages and so on. I think for the landing page, probably something that if

[00:18:17] Alvaro: you go, yeah, I mean, you, you could, you could totally use it. Yeah. It’s just basically a JavaScript library, so you can initialize it, whatever you want.

[00:18:24] Michaela: And then it’s made mainly about the look and feel. Yeah.

[00:18:28] Alvaro: So yeah, what it makes is that allows you to create this snap scrolling experience, uh, full screen. So, um, well of course it is much more complex than that. It has many more options that you can configure. You have hash URLs, you have history back and forth.

You have the lazy load, you have, uh, play, um, pause of media elements. You have many more. Right.

[00:18:49] Michaela: And I think this is it, right? So first you think, oh, what is it? Right. And it’s really small. And now I’m like, I have this blog and I’m thinking like, if I’m in my blog and I mark anything, then I would like to pop up that says, share it on Twitter and so on.

And then realized.

[00:19:04] Alvaro: Yeah, exactly. So the very basic functionality seems very easy, always to implement by yourself. And then 20% of the extra features that you want. Those are what takes the most time. It takes a lot

[00:19:17] Michaela: of time. Yeah, exactly. Like, and then you want to have like previous, a previous RT connects article, and then this is not good enough because you wanted by category.

Right. And so, yeah. So another thing that I was thinking a lot about and reminds me also of your success stories, and there are, I mean, there are similar success stories. I think now more and more popping up right around the internet is that 10 years ago, If you ask somebody to pay one Euro or dollar or whatnot, or 10 for something, they would like crazy.

I mean, everything was free and it was like software should not cost any money. And I think, especially in the last two years, three years, it completely changed. I [00:20:00] mean, if there are like Google. You know, sunsetting, most of them products that are for free, right. And then the ecosystem completely changed into this SAS businesses.

But also that you have more and more smaller, I would say, yeah, smaller, independent developers also really making money for their, for their software. Right. And suddenly I feel like, okay, we have to pay for everything. It’s getting really expensive. Like for my website, if I don’t want to use Google analytics.

Right. Which is. And this is like the, you know, 10 years ago, mindset, like let’s use Google, Google analytics, and now the mindset is, oh, we don’t want to use Google analytics. So what else do we have here? And then I find like 10 different analytics platforms, but they’re all. $20. And I’m like, okay, this is for my private block.

It, you know, it’s already, it’s already a lot, right? Because it’s not only the analytics that you need, you need a lot of things, but I see that people are starting to value software more. What’s your experience?

[00:20:58] Alvaro: At the beginning, it wasn’t common at all to transfer these things. And now it’s getting a bit more common, right?

But still on the JavaScript environments, especially in the front end side of the offense is not yet to come on to transfer these kinds of things. You see libraries, components of all kinds, but usually they tend to be free because I think one of the main reasons is it’s not easy to. Protects in some way, like the code is it’s free.

Everybody can see the code when it’s front-end. So it’s very difficult to protect something like that. And when you can not protect it, I guess if that’s the make or some people think it doesn’t make much sense to charge for it. But yeah, I know a few libraries that are starting to charge for demonstrable libraries, but are still not very common.

I think it’s more comfortable for other kinds of products for backend staff for kind of like subscription services. But I think it’s sending to implement. In that regard, because I think it’s been a, so like sometimes when you don’t see this, when you don’t see that the owner of the, the great of the component is [00:22:00] getting some benefit out of it, then it’s easier to see projects in GitHub that.

Get unmaintained or too, you know, that they will eventually die or get obsoleted or whatever, because you know, somebody can not keep a mundane enterprise for some people think that you can create an open source project, tablets it, and forget about it. But the publishing part is just the very beginning.

Then you have to keep on maintaining it and that’s for years, like my component is more than seven years. And you have to keep on improving it and adding features and reporting and dealing with reports. And you know, so many people don’t bother doing this. And then you see the, uh, the, you know, the last release was four years ago.

So it’s not really a component that sometimes you can trust or you have to look for an alternative or you open a, you’ll never have the answers.

[00:22:46] Michaela: So your experience with, you know, there’s this rumor I’d say, or the anecdotes around customers and especially. You know, the free riders, they they’re really like heavy maintainers customers that are asking a lot of things.

And then the people that are actually paying are, are much more moderate. And I mean, my experience is definitely like this. Whenever I provided things for free people come with this attitude that, you know, it’s there, it’s there. Right? Yeah. I mean, sometimes I get emails. Like I have a newsletter and you get an ebook, right.

That I wrote, if you, if you, for my subscribers. And sometimes they can’t find their email in the spam, right. It goes in promotion or spam or whatnot. Right. It’s not really my father’s and thing I can do. And I’m always, if somebody is, you know, writes me an email back and asks, you know, I didn’t get it. I’m always going, doing the wig to write them a personal email, send them the book.

Right. But there are.

[00:23:44] Alvaro: Yeah.

[00:23:49] Michaela: And for other things as well, what’s your

[00:23:51] Alvaro: experience here? Yeah. You always get some people that is a bit aggressive a bit, uh, you know, they want everything for free when you to work for them [00:24:00] for free doing consultancy, you fix their bags that sometimes they are not even related with your own product.

Right. So, well, what I used to me do is just reply to them. Very politely, politely explain them the situation. Um, you know, always telling them, have you found a bag or you are not happy with this because you didn’t pay for it. You know, then you can just go and check the code because it’s open, you can check it yourself and does it go same?

Like everybody can check it and things back if they find them. So I think, uh, how many, this, an open source is also a way to, you know, move people in certain that I send and tell them, Hey, this is open source, you know? You know, you’re taking advantage of it, but it’s also good because you can fix your own easiest, right?

If you have them. So that’s what I do sometimes. And all the times you just tell them, well, if you need my help, you can always upgrade to a, you know, the business license, and then I’ll provide you whatever support or whatever. So I give them the options and that’s. Yeah. You only did some, some angry people, the money so much, if they’re angry and they paid for it and they are not happy because there’s a bag that I cannot fix.

For example, then I’m happy to refund them. You know, if I can not find, uh, fix their bags, you know, that, that makes sense to me. But otherwise, if they’re not paying anything, then, you know, I help them to, to some extent, but not, you know, there, there are some limits.

[00:25:13] Michaela: Yeah. So how much of your work day now is really?

Are you still. Program or is still a developer in your mind or are you now, uh, you know, like you are a salesperson already or, you know, like an admin person. Yeah. You know, how do you feel like in your, in your entrepreneurial journey right

[00:25:33] Alvaro: now? I don’t know. I think I like to consider myself more. Person than a developer.

It is true that another developer and, uh, you know, the background is, and I keep on developing the product. Uh, actually I’m about to release a new major residence. That’s it is also true that I don’t spend. My whole time developing anymore. Like, uh, I just have to do marketing. I have to do content marketing and I have to do, [00:26:00] you know, taxes.

Uh, you have to do all kinds of things like images. Uh, you have to think about potential new opportunities. You have to think about the license system. You have to think about. I don’t know, uh, so many kinds of things. So yeah, I guess I, I like to consider myself more as a, as a business person. I would like to maybe eventually in the future delegate parts of the developing side to somebody else that is even better than me.

And so also there are people. Benefit from that, you always try to look for the best outcome and sometimes it’s also gets into readable yourself for certain tasks.

[00:26:34] Michaela: Yeah. Yeah. True. Yeah. So I don’t know. Um, I couldn’t find anything about, um, how much you’re making per month or per year. I’m also not sure if you’re sharing something publicly.

Some people do some people don’t, you know, I’m totally going with whatever you decide. Do you share it? Do you say how much you’re making with it or can you give us

[00:26:56] Alvaro: in my case? I prefer not to be very open about it just because, you know, it’s open source, so anybody can just see my code and create another project.

And I started doing the same, so, and that’s one of the reasons if I had the SAS, you know, a vacuum product or whatever, I wouldn’t mind, but right now I prefer to be more primates on these aspects. So, uh, you know, we can say that I get an app to, to leave. And that have different sources of income. That’s an, all, everything comes from from the main component and it also comes from the plugins for, for WordPress, from AFI.

The links that I have for selling other WordPress themes from support is. From consultant services from, and now I’m starting a blog as well, which I’m also expecting to monetize in some way. So yeah,

[00:27:41] Michaela: it’s good. Yeah. Yeah. And I think it’s really good to have multiple income streams. I also try to have more than one, at least for me, it gives me a little bit, um, Ease of mind that, you know, if something dries up or doesn’t work out so well, maybe SEO doesn’t work or, you know, something else comes a copycat, as you said, right.

You have [00:28:00] other opportunities as well. So yeah, I can totally understand that. Cool. So I think I probably, there are a couple of listeners that. Are also like one baby more freedom, or at least, you know, a side project that gives some side income. What’s your advice for them? What would you do like now you’re in that game for a long time or is it seven years?

What do you think? Is there what’s the right time right now to start

[00:28:25] Alvaro: something you’re saying, oh, the right time, the right time is always there because some people are still waiting to finish that course to finish that book to become better. And, uh, you know, to master a certain technology, I don’t think.

The way you have to take this the way you have to do. You know, do your best, be wherever, you know, and you will probably end up helping somebody. When I created my project, I wasn’t an expert developer in jQuery. I created it because I wanted to keep on learning and creating something useful was what was a good motivation for me.

And I think that’s what the people have to do. Well, yeah, I haven’t the same when I created my first website, I do it. I created it using Microsoft. I didn’t know how to code. I didn’t know anything about websites, but I had a teacher that taught me how to make websites used to Microsoft word. And then I started seeing people using the website and that’s what motivated me to keep on learning.

Right. So the right time, I think it’s always now just do whatever. Keep on improving it, keep on iterating. It gets feedback from people and they will tell you sometimes how to improve things, what features they will need to add, what bags they found. So I think that’s what really. To improve my components at first.

Um, what I would say is don’t quit your job, do something on the side, see how it goes, get feedback. And then when you get some traction, you can decide to quit. In my case, I only decided to quit. When I saw that I was going to be able to keep on living from. Took me six months can take more, can take less.

It depends on people, but I was working in on the side for three years. That’s what I would recommend people to do it on the side, if they [00:30:00] can not get too crazy because you know, a great idea might not result in people buying your product. So, yeah, that’s one of the things. And another thing that I think helped me a lot at the beginning was making it well, I was open source or let’s say.

Because when you have something for free, at least at the beginning, or you have a freemium kind of product or something, you get much more exposure. Like people will blog about that. People will share it on Twitter, on Facebook, or so you create this kind of like snowball where, you know, the content keeps on spreading faster because it’s.

Unless you have a very good product and you can already charge for it or whatever, having it free at the beginning might be good because you not only get much more free marketing because you know, people don’t mind certain staff when they know that it’s free and nobody’s going to really benefit from it.

And you also get the feedback from the. The more people, they use it, the more feedback you get and the more you can improve it on different iterations. And then, you know, in the future, you always can start charging for it, release new virus zones or, you know, or even people to another area or whatever, but having that free feedback and that’s free marketing, I think it’s very powerful.

[00:31:12] Michaela: Yeah. Yeah, I totally agree. Yeah. Thank you so much for taking the time, talking with me about your journey and, um, about how we could, uh, leverage open source to actually start our own little business and, uh, you know, get

[00:31:25] Alvaro: independent. Thank you for you for having me here.

[00:31:29] Michaela: Yeah, it was really great talking to you and, uh, thank you so much.

[00:31:34] Alvaro: Okay. Bye. Bye.

[00:31:37] Michaela: This was another episode of the self engineering podcast. If you enjoyed the episode, please help me spread the word about the podcast. Send the episode to a friend Wyatt, email, Twitter, LinkedIn. Well, whatever messaging system you use or give it a positive review on your favorite podcasting platform, such as Spotify or iTunes.[00:32:00]

This would mean really a lot to me. So thank you for listening. Don’t forget to subscribe and I will talk to you in two weeks. Bye.

Are happy developers more productive?

Are happy developers more productive? Let’s look at some research together and explore whether happiness and satisfaction affect developer productivity.

How to write tests that find bugs

Dr. Mauricio Aniche explains how developers can write tests that find bugs. Writing effective test cases that bring confidence is not that easy. Many developers are familiar with testing frameworks and regularly write automated software tests. Yet, often they don’t test systematically. That’s why Mauricio writes a book called “Effective Software Testing” which shows practical approaches to systematically test software. You can win a digital copy of the Effective Software Testing book, when you like and retweet this episode’s link, which you can find here.

We talk about:

  • the difference between effective and not effective test cases,
  • how domain-based testing helps to write good test cases, 
  • how you can use coverage reports to find missing test scenarios,
  • how to write testable code 
  • and about the importance of maintainable test code.
Mauricio Aniche

Today’s episode is sponsored by Codiga, a smart coding assistant and automated code review platform. Try Codiga for FREE!

Subscribe on iTunes, Spotify, Google, Deezer, or via RSS.

Transcript: Effective software testing 

[If you want, you can help make the transcript better, and improve the podcast’s accessibility via Github. I’m happy to lend a hand to help you get started with pull requests, and open source work.]
 

[00:00:00] Michaela: Hello, and welcome to the software engineering unlocked podcast. I’m your host, Dr. McKayla, and today I have to pleasure to welcome Dr. Mauricio’s Aniche to talk about systematic software testing. 

But before I start, let me tell you about an amazing startup that is sponsoring today’s episode: Codiga. Codiga is a code analysis platform that automates the boring part of code reviews and lets you merge with confidence on GitHub, GitLab and Bitbucket. I’ve worked with Codiga for around one year now, and I love how it also guides me in discovering and improving, well, the “not so nice parts” of my codebases. But there is more, Codiga also has a coding assistant that helps you write better code faster. Find and share safe and reusable blocks of code within your favorite IDE on-demand while you are coding. Learn more at codiga.io – that is Codiga.io.

But now, back to Mauricio.

Mauricio is currently setting up the internal tech academy of Adyen to better onboard developers and help them grow and upscale their skills.

In addition, Mauricio is also an assistant professor at the Delft University of technology. The reason why I invited Mauricio here with me today is that Mauricio writes a super exciting and very practical book about software testing. And he agreed even to give away one digital copy of his upcoming Manning book to one of our listeners.

So you asked, what do you have to do for that? Right. Well, there’s the chance to win this book. If you like and retweet this episode’s, tweet, I will put it in the show notes, or you can find it on se_underscore unlocked on Twitter. For an extra chance to win the book, you can add a comment on your personal best testing practice. But now let’s welcome Mauricio. Mauricio, welcome to the show.

[00:01:13] Mauricio: Hi McKayla. Thank you so much for the invitation. It’s a pleasure to be here.

[00:01:17] Michaela: Yeah, I’m super, super excited. I’m so excited to learn about this book. I have a copy already here with me, right? So I’m following you actually, while you’re writing it and you’re really far already.

And so I have a lot of questions and I think we can, you know, look at systematic defective testing throughout this episode and we can really deep dive and yeah, I can learn a lot and my listeners can learn a lot. So maybe the first question that I have is what. Writing tests in a systematic way actually mean to you?

What is that? What is systematic software tests?

[00:01:54] Mauricio: That’s a super good question. Mckayla, anything. That’s what makes my book different from the others? [00:02:00] So I think we are now super good at writing tests and we’re super familiar with tools like J unit and et cetera. But when it comes to write a test cases themselves, a lot of times we just follow our gut feeling.

No, our experience of course is super nice when, you know, learn a lot about what crushes a lot. So we write tests for this, but this is not really systematic. So what my book tries to do is to discuss techniques where you can try to create those test cases in a systematic way, kind of if you follow this recipe, Maybe the recipe will help you to come up with the interesting test cases that you need.

So this is more or less the whole story of the.

[00:02:45] Michaela: So when you say systematic and you know that we have to learn it, I’m thinking back at university. And the first thing that pops into my mind is fussy testing and, you know, and maybe verification and all of that, which you know, is scary. And a lot of developers don’t do it and maybe.

That practical. It’s not that scalable. Are we talking about those kinds of things or are we more thinking about edge cases, for example, and you know, what, what is exactly the systematic way to write test cases? Is it, is it.

[00:03:18] Mauricio: It is not, I think it’s closer to the thinking of edge cases, as you mentioned, so really close to the way developers write tests nowadays.

So it’s not a big jump from what you’re doing now to what I’m trying to propose in the book. And so think of, you know, you have this method to test this method receives two or three parameters. My book basically says, you know, look to the first person. What is it? Is it an integrator? Is it a string? If it’s an integrator, maybe you have these cases to think about, then go to the second pyramid there, go to the third parameter, then try to see how those three parameters interact together.

So it’s again, super similar to the unit test you. Right. But now in a more systematic.

[00:03:57] Michaela: So, is it a little bit like a checklist for me that sounds [00:04:00] like a checklist or, you know even it’s code reviews, you know, we know that the most effective code reviewers are actually the ones that have a checklist, right.

Even if they are experienced developers, because it helps your brain to go systematically through the code and think about, oh, did I think about this, about that? And normally people don’t even read the code review checklist anymore. They just look at it and you know, oh, I forgot actually about. That part over here.

Can I, can I think about it that

[00:04:27] Mauricio: way? Yeah, definitely. And to be honest, at some moment when I was writing the book, I considered a lot creating a checklist where people could, you know, just try to follow and these ideas are not fully mine. Right. So I learned a lot from what is in there. And so the domain testing techniques that are there maybe super popular in academia, a bit less popular in industry they are doing these checklists without calling them checklists.

Right? Yeah.

[00:04:52] Michaela: In the book you described quite a few different types of testing, right. Domain testing, which I would really briefly describe as breaking down the requirements. Right? So what the program should do and looking at that, then you talk about structural testing, which is looking at the code coverage.

Then you talk about example based testing property-based testing and you talk about contracts. Can you explain each one of those a little bit more in detail and maybe different. What is the most important one? Is there, do we have to know all kinds of this kind of different testing types? Or is it enough if I, you know, do domain testing, are they exclusive or how does the, you know, how do they integrate with

[00:05:35] Mauricio: each other?

Yeah, good question. I think, you know, if you’re a busy person and you cannot read all the chapters, if you read the domain testing one, and then the structural testing one, you can of. The main ideas there. And my suggestion is usually you implemented some feature, right? I’m not really focusing on test-driven development here.

So just imagine you wrote your feature, maybe you did test-driven development, maybe not. And now it’s time that, [00:06:00] you know, for systematic testing, you want to explore all the possible edge cases to look for bugs. My first suggestion is for you to do domain testing, meaning you look at the required. You know what the program needs to do, and this is the requirements.

It doesn’t have to be UML or a user story, whatever you like. And then you systematically apply domain testing techniques there to come up with test cases. So, you know, those are the inputs of my program. These are the outputs of my program. That’s explore. After you do this, you create lots of test cases.

Are you done almost the next step is let’s now look at the source code and try to get some hints from the source code, because you know, sometimes there are small differences between what you envision the program to do and how you implement it. Right. Maybe, I don’t know, you’re doing some crazy optimization there that require some extra testing.

Whatever. And then you look at the source code, maybe use some code coverage tool to help you on doing this. And you then augment the test suites you created with the main testing. And this is kind of the main loop that I see when it comes to create test cases. Then of course, after that, my book dives into different techniques you can do to, to make her tests better.

Like property-based testing, design the contracts so on and so forth. The two techniques are the ones that I consider core for someone that wants to grade group tests.

[00:07:22] Michaela: So domain testing, I have to see a little bit. Yeah. It’s not, it’s not black box box testing because I know the code. But it’s a little bit more about the use cases, right?

So maybe I’m looking at the JIRA ticket or, you know, some, some story in, in, on get hub or something, then I’m reading down what this, you know, what this code change actually should do. And then I’m developing my test for that. And then I should compliment that. With structural testing, we’re actually go at the code and see, you know, how it, how is it actually implemented and dry?

Because code coverage for me would be not only about [00:08:00] how is that implemented, but also like line coverage. Do I, you know, do I hit every line, even, you know, the exception and you know, if they are like, if there’s a switch statement, do I go through each of the cases and so on? Right. Is that, and then I go back to domain testing again, what should this actually.

And blew through that.

[00:08:18] Mauricio: That’s a super good point. And I think this is one of the things my book does differently from the others. And I’m super curious to see how the community views. So when we talk about the main testing, we usually have this black box perspective where we don’t care so much about the implementation.

What I’m trying to say in my book is that maybe you can use those same techniques, but now looking at small units that you kind of know how they work. You know, you don’t have to test the whole program using the main testing. Why not? The same things for small methods where a class, and of course you can then start going up in the test level.

So maybe you test the unit using these and then you’re satisfied. Then you go a little bit bigger and you test the whole component using the main testing. Why not? So I discuss the different test levels as well in the book, but so I want to first thing convince people that the main testing can be done at any levels.

Those techniques applied there as well. That is the first thing. And then the second thing is about code coverage because there were then usually focused on unit testing. And I also want to say that maybe it can help you in other levels as well. And, but I think the main difference from, from what I seen there is that code coverage for me is not there for us to see a number.

And, you know, did we reach our magical number code coverage for, for the CIA to pass? No, it’s more like it’s a tool to augment the tests you created using domain test. So they don’t care about the number anymore. You care about looking at a line that is not covered in reflecting about it. If in the end you achieve a hundred percent, I don’t care.

I just cared that you reflected about things you didn’t test. Okay.

[00:09:52] Michaela: And so, yeah, I know what you mean, but so one of the biggest criticisms for code coverage is [00:10:00] that we are, we are counting or most of the programs actually look. Has this code been executed or not, but we are not looking for.

Assertions for example, or like preconditions post conditions. So for me, they always go hand in hand somehow. Right. So when I think about testing and especially if I hear code coverage and coverage reports and this number right at the question is always okay, well, did I hit every line, but did I even check, you know, what’s happening here or that just execute the program, right?

Because well, if, if you know, a smoke test could also execute a lot of the code base. Actually just, you know, make sure that the code run somehow without verifying that, you know, the inputs and outputs are matching and so on. What’s your take on that. And what is your advice? Is it something that you, you know, that you talk about in your book and the developers should be worried about?

[00:10:54] Mauricio: No, I think that’s, that’s common sense in industry. A lot of developers really hate code coverage. And in fact, I have an entire section on the book called something like, why do developers hate coverage? Something like that. And to me, I think if you’re looking at code coverage as a number only, and then you decide whether you should go up or down, that’s, that’s not useful, right?

Because the number hides a lot of information. So for me, the first way you should use code cover. So again, augment the test cases you created before. So their test should always come from requirements because you should test what the program should do, of course, and then code coverage as a way to document your test cases.

And if you’re using this tool to see if you missed the test case, then code coverage becomes always relevant, right? Because you’re not only looking at, did I cover this or not, but it’s more like I didn’t cover this line. Why is that? Why did I miss this test when I was looking at the requirements? Right.

So it’s more of a reflection. Okay. In my opinion. This doesn’t mean though that the number is completely useless. I liked the code coverage. Number two gives you an overall impression of [00:12:00] how tested your system is because achieving 100% coverage doesn’t mean much because you still can have better. But having like 10% code coverage means something.

It means you’re not really testing anything there. So if you’re there to prioritize where to start, you’re testing the process right now say you’re in a legacy without tests. The code coverage can be a nice guidance for you. So that is the tip. Don’t use code coverages, a number that you blindly try to achieve.

That’s not going to work for you.

[00:12:30] Michaela: Yeah, I liked it very much and it reminds me of. You know, a study that I did during my PhD. And, and the tool that I developed it was actually called test hound. And did you static and dynamic analysis, right? To go through the code and to help develop. I understand the test cases, right?

So again, it was a comprehension tool which I think you’re, you’re using, you know, code coverage and those tools that are out there a little bit to help people comprehend. What’s actually happening in my test suite. How much of my code is actually covered. And then you go in right into for me what I did at that point.

It wasn’t looking at unit, but I was more looking at integrating. And I want to talk about integration testing with you anyway. And so what, what this tool did is it was showing how, how good integrations were covered or which ones were not covered. Right? So again, so a little bit showing the blind spots.

And for that, I was really comparing. Dynamic execution, you know, and, and stabbing execution, because then you could overlay that and say, well, actually, in the dynamic setting through testing, we didn’t cover that part. Right. So yeah, I like that very much. How do you, how do you think about integration tests and system tests and, and unit tests and, you know, there’s this th the testing pyramid, which I think is very, very common and very well known, but is it still valid nowadays the pyramid has like biggest part is the base, right? And then the higher we go up the [00:14:00] perimeter a little bit, you know, reduces its size until, it’s just a little, a little spot at the beginning. So and then each of the levels for testing is actually reflected in this pyramid.

So it more or less says that we should do focus on unit testing the most, then a little bit less on integration testing a little bit less than system testing, testing and so on. Is that something that you recommend is that a, is that the view that you see.

[00:14:26] Mauricio: You are only asking me the tricky questions Mikayla, before I answered the test pyramid, let me make a remark about twos, because you mentioned you developed the tool and et cetera.

And I feel it is about time for us to take the next step when it comes to code coverage tools. Because right now I feel most of them are super interested in reporting a code coverage. But they are less interested in using coverage to help guide developers. What is the next test I should write? And if few, the first company person that does this will have the code coverage tool that will be used in uh,

And then , , you mentioned about the testing pyramid indeed. So the base of the pyramid is the unit testing and the base of the pyramid is a bit larger because you should do more of these, right? That’s the idea of the testing pyramids and you see out there, some people love the testing pyramid.

Some people. Not to focus on unit tests, but focus more on integration and system tests. I belong to the team unit testing. So in my book, I do recommend people to try to focus on unit testing as much as possible, the reason being I believe that if you design your code, well, the core of your system, the important parts of your system will be just a.

For loops and F’s and data structures being manipulated. And those can be easily tested with unit testing and by easily, I mean, it’s super easy and fast to write a test, they run super fast. You can quickly explore different corner cases. You know, it’s super easy to just instantiate a class, put some values in column methods.

That is why I prefer unit [00:16:00] testing. But that requires though that you develop your system with this, you know, unit testing, the stability in mind, and this is not always. Why do some people prefer integration testing and they have a point there because, you know, in lots of types of systems, we do a lot of the bugs only happen when you put components together.

Right? So think of a distributed architecture, maybe microservices, whatever those components are, exchanging messages. And then maybe, you know, a crash will only happen with certain combination of things. And if you’re really mocking out components, you know, when testing one component, you kind of mock the rest, maybe you’re going to meet.

The cool stuff. Right? So that’s why some people like integration testing. And to be honest, I think we should do all of them. My suggestion is focus a lot on unit testing, ensure that your small units, small components are super well tested via unit tests, and then go for integration testing, maybe even system testing to explore things you cannot explore with unit.

Those are more expensive. So maybe you have to prioritize a little bit more, which test cases you’re going to write with all the levels are important. Then the challenge is to find when to write which type of tests.

[00:17:17] Michaela: And is your book going into that a little bit? Is it giving some guidance for developers on when to ride?

Which tests do we get some external?

[00:17:25] Mauricio: Yeah, it does. Right in chapter one already have a huge discussion on the testing. And then I say, you know, based on my experience, when I go for unit testing, when I go for integration testing, and then in chapter nine. So one before the last chapter of my book is all dedicated to integration and system testing.

So yeah, that’s good. There is a bit tricky, right? Because when it comes to integration, testing, and system testing, To me, highly contextual. So if we’re developing a microservices type of application, you may have some best practices there. If we’re developing a mobile [00:18:00] application, maybe you have another set of practices.

So I tried to make a more generic discussion, you know, on integration testing.

[00:18:07] Michaela: So you will also talking about something that I want to explore a little bit at that testability. Right. So you were saying, well, it’s a little bit harder to make things testable and, and, you know, you have to have testability in, in mind.

What does testability even mean and what did design principles, software design principles help us make software system more testable on different levels? Right. So is there a difference in making unit tests more testable versus integration and system tests or,

[00:18:37] Mauricio: yeah, a good question. I think. The testability rules, if I can call them like that are not so complicated, but it’s super hard to apply them.

Once you start to develop more complex software, because your mind is just full of other things to think about. But a testable system is just a system that is super easy for you to control. So I want to test this specific class or a specific component of my system is super easy for me to tell the component what I wanted to do.

And it’s also easy to observe. What this component is doing. So laughter the component is done. It’s easy for me to assert that the component behaves as I expected. So those are the two things that you should always have in mind to design testable systems. Now when it comes to source code and implementation if we think of Java code, for example I think rule number one, when it comes to testability of Java code is your classes depend on other classes.

You should be able to test your class without really depending, too much on the dependency. So we want to be able to mock some dependencies maybe so that you, you gain some control. And so I feel like the most common anti-pattern I see there when it comes to testability is, you know, you’re creating a class.

Make this class to depend on five other classes, but you don’t give away for the testing codes to control these things. And this starts to accumulate, and this becomes like a snowball and [00:20:00] it’s super easy to lose control and then to never be able to write unit tests anymore. So that is rule number one.

There are many others. I discussing a book, so I’m a big fan of the ports and adapter. Execunet all architecture. I feel like they all go in line with the way I see design for testability. So it’s not super complicated, but I feel we need to put a lot of efforts in making sure we are never forgetting to do these things.

Does that make sense to you? Yeah, it makes

[00:20:30] Michaela: sense. The question is a little bit, okay. We can create mocks. We can stop things. But first of all, it, it, it takes a lot of effort right. To do it. And, and then if we do it, like, for example, I am writing a lot of data intensive applications. Right? Very, you have like a lot of, a lot of large data, you know, inputs that are done somehow processed.

And if you want to mock that, or, you know, Parts of it. It really gets very tedious and it gets hard to control. And especially if you’re, you know, like, let’s say we are mining data from GitHub, right. One of the things that I love to do so we are, we are, we are connecting to the GitHub API. And now we want to make our test cases, obviously independent of that, because we don’t actually want to query, you know, those large data sets for every test that we’re running.

So we have some version of that locally. And it gets really complicated. Like not complicated in, it’s hard to do way, but it’s very tedious to do it. It’s error prone. And then, you know, the ABI changes or the format of how they present, you know, the XML structure and so on changes. And then you have to go to your test code into your marks and your stops and your, you know, dummy data.

And, and change all of that. What can we do to, you know, make our lives a little bit easier here? Is there something, or is it we have to deal with it?

[00:21:56] Mauricio: Yeah. Good question. So I feel like if it’s becoming [00:22:00] boring for you to mock stuff, this is just the code telling you that there’s something wrong, right?

It shouldn’t be hard to. So usually my decision-making processes, if things are hard to mock, are there, is there any other way for me to redesign things, you know, maybe move some responsibility there, some other responsibility here so that the design of classes make mocking a little bit easier.

That’s number one. Number two is I feel like sometimes we have this idea that if we’re using mocking, we should only test one class and then. And that’s not really true, right? Maybe you can test a bunch of classes together and just mock the classes that are, you know, super hard to use the concrete ones.

So you can test higher components and some classes are the, you know, the concrete implementation during the test, some other classes you just mock. So I feel like you can also try to, you know, go one level up and look at, you know, classes together rather than one single. That being said, some domains just are not unit testing friendly.

And the domain mentioned mining reposts is it’s one that is super close to my heart as well as a researcher. And so when I’m doing these type of tools, where I extract data from V3 post threes, or I bar source code to come up with code metrics I don’t try to unit that so much. So I have a, I have a project in my get hub called CK, which is the code metrics tool.

It looks at the Java class and tells you the complexity, coupling and et cetera. I don’t mock. Because it just makes no sense. It was super hard for me.

So 90% of the tests there are integration tests. So there’s also, you know, these type of domains where you should go for integration tests because they will pay better.

[00:23:37] Michaela: Yeah. Yeah. I think that’s a very pragmatic answer. I like it very much. I also, in this, in this kind of situations I haven’t. Rely on integration tests, but I actually also ride a lot of unit tests for things that are easier to unit tests.

But if I see that it’s really hard now, I change my method and do something that I feel, is a little bit [00:24:00] more meaningful also from a maintains perspective. Right? So. Maintain ends, I think is also something. And maybe something that I want to talk with you about tenants, I think goes hand in hand with Tesco because we have to maintain Tesco, right?

If you don’t maintain it, and if it’s really hard to do it, we are not doing it. You are disabling the tests. I know it. I did it right. I At one point, you’re just giving up on it and to say, okay, already commented out or I deleted, or the worst thing is common to the alt-right.

So I think Ventana, the tests are so important. Do you have some tips for us how to get to maintainable tests?

[00:24:32] Mauricio: My book proposes 10 guidelines. I of course don’t know them all by heart, but the, they are all about readability, maintainability of Tesco because in practice you’re going to write lots and lots of them.

So my generic advice there that encapsulates those 10 recommendations in the book is you should pay as much attention to the test code. As you pay to production codes. We love to refactor. We love to create abstractions. We love to isolate the code. The same techniques you can apply in your test code and that will pay off for sure.

Finally, Michela, you mentioned you mentioned flaky tests, those really happen. And we should try to avoid them as much as possible. Although I just wrote a blog posts on the website of my book saying that I don’t believe we can ever get fully rid of them, because if we’re in a super large and complex system they will happen.

And sometimes for reasons, you can’t control for reasons you don’t fully understand. So I feel our job is to not only avoid them when possible, but also to keep them under control.

[00:25:41] Michaela: One thing that comes to my mind is when I was working at Microsoft what we did, like windows had flaky tests, right? Especially because the integration tests and the functional tests where we really try, you know, under different settings, the systems and so on.

Blah, blah, blah. Right. So it gets really complicated. It’s really, you know, a lot [00:26:00] of configuration, so you’re running a lot of tests and some of them just fail. And so what we did here is we, instead of, you know, sad, well, let’s, let’s disable all the flaky tests or, you know, it wasn’t even possible to fix all of them, right.

Because it was sometimes. Clear why there, you know, why they’re flaky. And sometimes what we did is we measured their effectiveness. So how often is it flaky tests actually finding a thing? And how often is it giving a false positive? And we had like a cost benefit analysis where we say.

Well, every time it fails for no good reason. It’s a cost of your career because we have to go and investigate and find out actually nothing happened. And then for every time it actually finds it back. We tried to estimate. You know, how much did that actually help us now? Right. So how much how much did we safe and so for each of those tests, we had a profile on, you know, from past runs and we looked at that, I mean, it was again, obviously an automatic approach.

But again, test case comprehension. So what we provided office, for example, with, and we worked with windows was these are your test suites, right? These are the test cases. And this test, you know, is more red than green, more red than green means, but it’s failing really often. So you have to investigate really often and, you know, you just realize nothing happened.

And it actually doesn’t really find anything. Right. Or the other thing is, well, it fails quite a bit, but it also finds quite a lot of things. So we still think you should keep it or, you know, go and improve it and so on. So yeah, this is something that came up from my mind when you were talking about flaky tasks is that something that you have ever looked at?

[00:27:41] Mauricio: Yeah. So all big software has flaky tests, right? And agile is not different. So there are some flaky tests there. And the approach that the engineers have is super smart in my opinion. So they first detect the flaky tests. If the test is freaky, the first thing that happens is we move these two separate.[00:28:00]

Right because we want to keep, you know, one test suite free of flaky test. So this one we can run and we can trust. And then we have a second test suite where we, we have the flaky tests, so there, and we keep trying them more than once. If they pass, we are somewhat okay. And without the developer, you know, this is flaky, but it’s sometimes specialist, please review.

We tried to see how often it is flaky. Right? So the developer has more information to understand it. But what I think is super smart is the separation, right? So we don’t want to, because if you have flaky tests and then you break the build because of a flaky test, you’re breaking the entire organization.

Right. And you don’t want to do this. So separating them from, from the tests that are not flaky, something super smart. The project does. Yeah. That’s very cool. Yeah. Yeah. So I feel like as soon as your software gets that big, you’re not going to be able to avoid flaky tests. So you have to learn how to live with them.

And I wrote a blog post about some smart stuff I saw not only at adjunct, but in industry in general, we have to get used to them. The more complex we do software, the more we’re going to see.

[00:29:06] Michaela: . I will link that in the show notes. So Maricio maybe the last thing that we want to let our listeners know again, is that if they go to , as the underscore unlocked on Twitter and retweet this episode, and then you have a chance to win this awesome book from a ratio about systematic software testing.

And if you add a comment about. Practice in testing software, then, you know, you’re a double in the pod, right? So there are two entries of you in the pot, so you have double the chances to win in the book. And so yeah, with that thank you so much. Mauricio I think it was a super great interview. I learned so much, I heard so much from you.

And I think my listeners as well, so thank you so much for being on my

[00:29:46] Mauricio: show. Thank you, McKayla again for the invitation. I loved it. Yeah, it

[00:29:51] Michaela: was really great. Okay. Bye. Bye.

[00:29:53] Mauricio: Bye.

How to build a strong engineering culture through engineering values

Learn how engineering values can help you build a strong engineering culture and empower your developers to make decisions that are aligned with your goals.

Measure developer productivity using the SPACE framework

Dr. Storey explains how to best use the SPACE framework to measure the productivity of software engineering teams. Dr. Storey is a Professor of Computer Science at the University of Victoria and a distinguished expert in empirical software engineering.

We talk about:

  • Productivity metrics for software developer
  • Developer experience as a different mindset to improve developer performance
  • The SPACE framework, which focuses on giving a well-rounded understanding of developer productivity. 

Book your awesomecodereview.com workshop!

Subscribe on iTunes, Spotify, Google, Deezer, or via RSS.

Transcript: Measuring Developer Productivity with Dr. Margaret-Anne Storey 

[You can help make the transcript better, and improve the podcast’s accessibility via Github. I’m happy to lend a hand to help you get started with pull requests, and open source work.]

[00:00:00] Dr. Michaela Greiler: Hello and welcome to the software engineering unlocked podcast. I’m your host, Dr. McKayla. And today I have the pleasure to talk to Dr. Margaret-Anne Storey. Dr. Storey is a professor of computer science at the university of Victoria and a distinguished expert in the field of empirical software engineering.

[00:00:18] I had to pleasure to work with Dr. Storey on many occasions. Even this year, she joined me during a research I led on developer experience, looking at what makes developers happy and productive and leads them to stay longer and more engaged in their job. Today, I have her here to tell us more about developer productivity and especially, I want to know how can we measure it? How can we improve it? And so I’m super happy to have Dr. Story or Margaret-Anne, or actually Peggy how friends call you. Here on my show. Peggy. Welcome to the show.

[00:00:52] Dr. Margaret-Anne Storey: Thank you very much, Mikayla. It’s great to be here.

[00:00:55] Dr. Michaela Greiler: In the last episode. I talked about my perspective on developer experience and developer productivity and well sort of, there was a little bit of a conclusion, uh, bottom line, which is that I think organizations and teams should stay right away from focusing on measuring develop. productivity and more think about developer experience. This means like, how do the developers feel about their work? Are they, are they feeling productive or are they happy? You know, do they feel that they make progress? Are they bothered by some tools and so on? But what is your take on measuring productivity?

[00:01:32] I know you did a lot of studies there, so you have a lot of expertise. What do you see? think?

[00:01:37] Dr. Margaret-Anne Storey: All of my research that I’ve done over the last few years has really pointed towards taking a lot of effort and understanding what you mean by productivity before you try to measure it. So what does productivity mean to you? I know that in your last podcast, you mentioned the space framework, and that is the summary of a lot of years of research that shows that, that there are many different dimensions to productivity.

[00:02:01] To some people it’s about pull requests or lines of code they committed or features delivered to the customer or ability to be able to learn something new or help other people. You can’t just say that there’s one way to understand productivity or define it. There’s many different ways and consequently, there’s different ways of measuring it as well.

[00:02:23] It’s interesting to say that we shouldn’t measure productivity, but the fact is a lot of companies will still try to do that. And I think a lot of developers also try to think about how productive am I being? Am I being as productive as I think . I really like this pivot towards thinking about developer experience, which is what we worked on together.

[00:02:45] And you invited me to join that wonderful project that you’ve worked on. And I loved that because a lot of my earlier research really distinguished developer satisfaction from developer productivity, they’re definitely related. So more satisfied, happier developers will be more productive and feel more productive and vice versa.

[00:03:05] But there is this difference. Developer productivity to me really it’s about measuring what they do, right. Or how they do it. The amount of work that they were able to do, or the amount of value that they provide, whereas developer satisfaction or developer experience is more how they feel.

[00:03:22] And those are two quite different things. So I really like that nuanced change that you brought to that through developer experience.

[00:03:32] Dr. Michaela Greiler: You brought up two things, which I think are also in my opinion, two extreme different things, which is the things that they do and the value that those things bring. I thought about this, especially in my entrepreneurial journey it even impacts me much more, so that everything that I do should also lead to impact and have impact and have value.

[00:03:54] Sometimes I see in literature that it’s productivity versus performance where performance, a little bit more output oriented. But even there, I think it’s really hard to grasp . So what is productivity? In my last podcast, I was comparing it to the industrial age where we really have off producing something which comes back to lines of code, and impact has nothing to do with lines of code.

[00:04:16] You can have one line of code in. it can have a huge impact. It can be a drastic bug, or it can be a line of code that really is such a satisfying thing for the customer. I think there are really a lot of nuances, and I wonder, people very often, I see them measuring productivity because it’s the easier metric. I actually really liked the SPACE framework. I’m super inspired by it, but I’m also a little bit skeptical on the metrics side of it. I think it’s so great to have it as a mental model to think about productivity in all those different variations. But then on the end we have like these metrics that are very isolated and then we try to stick them together.

[00:05:01] As I understand it. Right. You’re the expert here. So I I’m happy to hear much more about that, but so we should have different dimensions and take at least three or something. Um, but then in the end, like we were taking very different metrics and how are we going to combine them? What will the combination tell us?

[00:05:18] Can we even interpret it? So these are all questions that are racing up in my hat. What do you think about that?

[00:05:25] Dr. Margaret-Anne Storey: I’m not a fan of developing a lot of metrics, honestly. Let me just maybe step back a little bit and just remind our listeners what the SPACE framework is about in case they didn’t get to listen to your last podcast, which by the way was great. I loved your discussion. SPACE framework is very much an overarching framework to think about different dimensions of developer productivity. The goal behind it wasn’t so much defining metrics, but just thinking about when we talk about productivity, what are the dimensions about productivity that we may mean, or other people may mean. So SPACE is an acronym, and the first letter is S and that stands for developer satisfaction and wellbeing. If we think about understanding developer productivity or improving developer productivity, if we make some change, say to improve developers productivity, we need to think about the impact of that change on developer satisfaction and developer wellbeing. The second one is P which is performance. So performance or the outcomes or the quality of the work that you’re doing. And again, what performance means really varies from developer to developer, engineer to engineer, or managers. They all have different views about those performance outcomes that they care about.

[00:06:44] And then the last three dimensions, the A is for activity, which is what a lot of people think about when they think about development productivity. They think about lines of code. They think about pull requests. They think about features that are delivered to the customer. So that’s kind of the typical one that developers tend to think about when you ask them about developer productivity. C is communication and collaboration. In the research that I did with now, thousands of different engineers, when you ask them, what does being productive mean to them?

[00:07:15] For many, they say it’s about collaborating with others. How well I helped others and how well I collaborated with others. And that’s not surprising because software development is such a collaborative activity. You don’t write code by yourself anymore. And then finally, E which you also touched on is how efficient I can be. My ability to be able to get my work done without a lot of interruptions and my ability to get in that very pleasant flow state, so that I really feel immersed in the work that I’m doing. The work that I’m doing, isn’t so challenging that I feel overwhelmed, but it’s at that sweet spot of being challenging enough that I feel that it’s very rewarding.

[00:07:53] Arty Starr also talks about flow in her book called flow about software development, and talks about the joy of development, which relates again to that experience and developer satisfaction. The SPACE framework honestly is very high level because it impacts all of those things, right? It impacts how developers feel their satisfaction and their wellbeing, the performance.

[00:08:15] So the quality, the outcomes of the project that you’re working on, or the tasks that you’re doing. And then it also looks at the activities that you do and then how you collaborate with others. And then that efficiency and flow part, which again, relates into satisfaction and has an impact on performance.

[00:08:32] So these dimensions, they’re not stand alone dimensions. Maybe you make a change that makes developers feel more satisfied and their wellbeing goes up. So for example, you make a change that allows developers to spend one day a week learning something new.

[00:08:47] That could have an impact on their ability to collaborate with others. If they don’t all work on that same day, they won’t hear back from others and it might have an impact on their activity, at least in the short term. So you have to think about these together. In the SPACE framework paper that we wrote about, we spent a lot of time talking about these different dimensions and the fact that there are these different metrics. None of us, I think really believes that, okay. Take SPACE, choose three dimensions and then choose three metrics and you’re done. That’s not really the best way to use it. However, doing that would be better than choosing one metric.

[00:09:26] The benefit from space comes is thinking about these different dimensions and thinking about. What is it we understand about these different dimensions. What do you, and I understand about activity and what that means to have more outputs. When I say software quality, what does that mean to you? What does it mean to my peers on my team? What does it mean to my manager? What does it mean to my manager’s manager? And when I say that I’m spending a lot of time collaborating. What does that mean to you? And what does that mean to other people and so on. So really thinking about those different dimensions, realizing that any change in one is going to have an impact on the other dimensions and coming back continuously to reflect on those and to reflect on how do we each think about these and how do we each think these will be affected if we do make a change to try to improve overall.

[00:10:18] Dr. Michaela Greiler: One thing that came up also in the developer experience work that we did is this idea of between short-term and long-term. And I think, especially in measurements are falling short in that regard, right? With your example where you have wellbeing coming or going up because they’re spending one day learning and then two other dimensions going down because now they have less overlap for the collaboration and they are writing less pull requests or lines of code. There’s also another dimension, which would be maybe learning right. And learning then in terms of what’s actually the performance of the engineers? Do they create better things, maybe they are inspired and all this becomes really intangible. How do we measure it? Is it happening right now? Maybe it’s happening. We see this. If you measure the things wellbeing, let’s say we have a survey and we ask people and we see well-being goes up, then we see from Git, well, the commits go down. We see maybe collaboration overlaps go down. We can measure that as well, but what’s really hard to measure here is that, maybe somebody has an innovative idea. Or maybe somebody had this idea and learned something and this saved us from a bug or created the more maintainable software system because of the things that they learned here.

[00:11:35] This is a little bit my critique point here, and I really like your stand on it. It’s just a mental model to help get all these different dimension, a little bit more organized, because there are so many things floating around. So this helps us looking at the SPACE framework.

[00:11:50] We can at least go through some of the dimension and we’re not measuring it, but we make a thought experiment like I did right now and say we have a very strong feeling that in the long run our engineers will be more innovative. We’ll be up to date. They will learn. They will maybe stay longer with us. I know that for every job that I was stuck, where I felt like I’m not learning. I was very unhappy . And I think a lot of engineers are like this, they want to learn. so I think maybe it’s a nice mental model to think around those things.

[00:12:20] Dr. Margaret-Anne Storey: That’s a really good summary. I’m going to just mention some of the more recent work that I did with Brian Hopkin and Tom Zimmerman, where we looked at alignment between different developers and their managers in terms of how they define productivity.

[00:12:34] And we also ask them how they define software quality. And what we found is that there are many, many different views of what productivity and software quality mean. If you’re saying to somebody, should we use this new tool? Should we allow engineers to spend time learning?

[00:12:51] You have to unpack what are all of your assumptions about what the impact of that change will be on developer experience or the quality of the product, whether it’s in the short or long-term and literally expose all of those assumptions, but also expose what it is that you don’t know.

[00:13:11] If we make this change or introduce this new tool, what are the things that we need more information about? And often enough, I’ve sat in on a lot of meetings and people are using terms like productivity and quality, and it’s clear that they don’t even mean the same thing. And yet they’re in these meetings trying to make decisions, and these decisions are very strategic and the decisions are often made without really unpacking.

[00:13:37] We have comfort when we have signals of what’s going on, but software development is a very complex sociotechnical activity and you can’t reduce it to these very simple metrics.

[00:13:49] This came out in the work that we did too Mikayla, but not everybody’s the same, right? Some engineers might be happy not knowing what the impact is and they may be happy helping the people around them. And that’s what they really care about. I helped four people today.

[00:14:06] I’ve worked with people like that. They’re not egocentric at all. And they’re less focused on understanding the vision. They’re happy to help the people around them. There’s no single way to measure this, even if we use surveys as well.

[00:14:19] Dr. Michaela Greiler: There’s so many good things that you just said that I want to touch upon. why are organizations striving for measurements? the higher up, the more we want some tools. We want to understand the world. And I think it’s this idea of models. Of abstractions, because it’s just too hard to grasp. So if you are the team lead and you have like five engineers. You probably have a really good idea of how things are going.

[00:14:45] Are we productive. Are we performing and so on? If you then the manager of managers and you have a team of 50, I think it’s really hard to understand everywhere. Like, are we doing good? If you have an organization of 500 engineers, it’s like, I’m flying blind, right? I’m still really, really skeptical about building a model, which is so abstract that every model is wrong, that it doesn’t reflect the reality.

[00:15:12] And it doesn’t really help us. And I like what you said about, there are things that we know, but there are unknown unknowns? We don’t even know what we don’t know. And I think a lot of. Our activities in engineering or some of the else is about the unknown unknowns.

[00:15:27] We have to make decisions with the information that we have right now, to the best of our ability. And I’m really wondering if those metrics, if they’re helpful, or is it just that we feel that it’s helpful? Maybe it’s, I think it’s maybe helpful for somebody that has some agenda, right.

[00:15:43] So they want to come up and so now to make the metrics look great. And it’s, again, this short-term vision of, oh, I want to Excel here. I want to be the VP of engineering here. So I make this because it’s easy, right? And even if you have like five or six or seven metrics, we can tune them.

[00:16:00] We know that we can make people respond to it. We can’t show that this is really the outcome that we strive for longterm, but it enables us maybe for some hidden agenda. And it’s more personal than what we are actually going for this main goals. What’s your perspective on that?

[00:16:19] Dr. Margaret-Anne Storey: I think that a lot of organizations use metrics because it helps have a handle on complexity. If I can somehow measure what’s going on, then I can have confidence that the decisions that I made last week or last month or a year ago were the right decisions. Or I might get data back that indicates that those decisions were not right. And that we need to make a change. Good managers know that there isn’t just one metric. They’ll know that there’s a host of metrics. And I think really good managers and really good leaders will also listen and ask the right questions to find out more nuances, more deeper understanding about what those metrics mean.

[00:17:02] I’m not saying don’t use metrics. Use metrics and also have rich insights and be continually reflecting and revisiting your assumptions and thinking about, you know, what are are your goals.

[00:17:15] Are our goals to sell more of our product? Well, yes. Right? Because we need the money to be able to make sure that our developers are well paid. But once we’ve done that what else are our goals? Is it to improve the culture so that when people come to work, that they feel secure and they feel safe and happy and they have psychological safety.

[00:17:33] Is our goal to retain our really best talent over the long-term and, and how will we do that? Understanding those goals and then understanding which of these metrics from a whole different set of possible metrics, I think is one thing to do, but really looking at what do our metrics tell us.

[00:17:52] But what do they also allied? Right? What do they hide? What are they wrong about? What is our data gathering? What is our data not gathering? Every time we define a metric, what risk are we introducing by using it? And what else do we need to gather? Like maybe some stories or insights to give us that full picture.

[00:18:10] That’s why I think the SPACE framework is quite powerful because it helps us kind of identify a whole set of things that we need to ask questions about and that we need to listen to those answers, to be able to make better decisions in the future and make change.

[00:18:26] Whether it’s change to address a problem or change to make some kind of improvement. There’s a lot of different things to consider and it’s an attempt to get people to slow down and not to rush, to define metrics and then create a dashboard and then look at it once a week and say, look, our numbers have gone up, wait a minute what’s happening behind the scenes here.

[00:18:46] Dr. Michaela Greiler: So what comes to my mind when you talk about that is the goal question metrics framework ? And it’s a little bit of a different approach. I always say, do not measure productivity. You know what I mean by that is in this very simplified way. I think the data is so, so powerful. And I think those things are two very separate areas. The critique that I have for how people use it is that they are creating metrics. But they’re not data driven. I always felt like this connection because you had the same thought about data-driven investigations and research, which is we combine qualitative with quantitative. I personally really think that they’re only in combination, they become powerful.

[00:19:32] I think that only qualitative isn’t as powerful, only quantitative can be extremely misleading. This is the metrics area that I’m talking about in the net critiquing so strongly. But I think that if we combine them, this can be extremely powerful.

[00:19:45] In industry, it hasn’t really landed. It’s also quite complex to combine that to have mixed research approaches. And behind the research approach, there is also a hypothesis. Questions that have to be tackled and so on.

[00:19:59] I’m consulting organizations, where we look at their data because I think it’s so powerful and it can help us guide and make decisions in this very complex world. But I’m always missing this qualitative aspect that we actually go and say, what does this mean? And so coming back to code reviews, it’s turnaround time is on one hand super interesting metrics. On the other hand, it’s completely meaningless. If I take it as face value, if I built a dashboard and I’m printing out turn around time there it’s maybe meaningful for the first week. It only becomes meaningful if I do the qualitative work which means that now I’m digging deep and I’m trying to understand what’s happening here. Why I’m seeing this number? Is this even a number that I should take at face value?

[00:20:51] Dr. Michaela Greiler: Probably not most of the time, not. And I wonder how can we bring that to industry ? How can it be applicable and manageable for industry what’s your take on that?

[00:21:01] Dr. Margaret-Anne Storey: Oh, that’s such a great point. I think that there are a lot of cases of industry that do use mixed methods and do use qualitative data and quantitative data. I’ve seen some examples of this and I’ve seen some examples be used really, really well. I don’t want to mention any company or any person in particular, but I saw it with large companies.

[00:21:23] And I saw one colleague in particular. He went and he sat and he watched developers to see what their pain points were. And that was incredibly valuable to identify some really easy to solve pains.

[00:21:37] Dr. Margaret-Anne Storey: Another example, this was more research that I did with, with Microsoft and we published this in our Tripoli software or my student, Laura McCloud. She was looking, we were looking at the data and looking at the. And she was going around and following sort of jumping from office to office and seeing what happened behind the scenes.

[00:21:57] The code review tools collect all of the telemetry of what happens that the tool records, but it doesn’t record the engineer jumping over to somebody’s office and saying, ah, by the way, I just tagged you in a, in some code that needs reviewing. Do you think you could do this quickly for me?

[00:22:13] Or, oh, I see that you submitted this code and it’s got this really big problem with it. Do you want to fix it before you really send it to me to review it? Because they don’t want to embarrass somebody by finding a big bug. So understanding what kind of happens behind the tools and behind the data is really, really critical.

[00:22:32] Sharing examples of this with companies and showing them why that’s powerful to have these rich quotes, these rich stories. about what’s going on to augment the data, to go hand in hand with the data, I think does shift how people approach it.

[00:22:48] Data only tells us what’s happening. It doesn’t tell us the why. It doesn’t tell us what we should fix. It doesn’t always tell us what is actionable in here that we can change. So we may see over time that engineering productivity, according to the activity metrics goes down. But that doesn’t tell us why it’s going down, necessarily. We have to observe, talk to developers to find out what’s going on here. Why are you not committing as much code as you used to? And then by talking to them, you can then get insights that help you make changes, and then you can use your metric to see, okay, is there a change? Do we see this change?

[00:23:26] Dr. Michaela Greiler: And I think especially with that one, it could be just a change of how people use the tool. Maybe they squash commits and we are commits

[00:23:34] Dr. Margaret-Anne Storey: Yes

[00:23:34] Dr. Michaela Greiler: Right.

[00:23:35] Dr. Margaret-Anne Storey: Yeah.

[00:23:35] Dr. Michaela Greiler: I think the, the biggest problem here also is that okay, if you have one team it’s normally quite simple, but if you have several teams and they have very different work styles, then, these numbers really become meaningless.

[00:23:48] I would love to find more ways for organizations to bring that insight . This ability to actually have metrics. But not only have metrics really have measurements. I distinguish between measurements, investigations instead of metrics. I define it once and then, half a year later, they are outdated. They’re not reflecting reality, but I’m measuring and I’m having data. I’m looking at data, I’m doing investigations . This investigation mindset. I think that would be so strong for organizations and the ones that I’m working with I see so many really good results. It’s almost like enlightenment ? Where metric is a light somewhere making a little bit, in the dark, something visible.

[00:24:28] Dr. Margaret-Anne Storey: The other thing that I’ve come across is that a lot of folks in industry think that interviewing or observing developers is very time-consuming and it doesn’t have to be, you can learn a lot from sitting with your team at lunch and asking them, what are your barriers? What are the challenges that you face? How is your experience. And gathering those insights just even once a month can lead to a lot of insights that you can then use to make change.

[00:24:59] Dr. Michaela Greiler: I think one perspective that I also want to add here is that this qualitative aspect that I’m talking about, doesn’t always have to be talking to people or observing people. It could be that there’s a person that has this task and they’re going in, they’re investigating, let’s say 50 pull requests. And they writing down what’s happening here, or the last three bugs, why did that actually happen? Some quality metrics or if you’re thinking about metrics again, metrics for me would be oh, line coverage. But then a person really doing the investigation going and saying, why is this project so different in line coverage than that project ?

[00:25:36] Or why are people having those large pull requests over here and not there? And very often, if we then do the work, and I call this also qualitative, because you’re not collecting data. It’s not quantitative. You have to look at it, you have to investigate and you have to see, oh, actually this is very coupled ?

[00:25:52] This is the reason maybe there’s a framework.

[00:25:54] And metrics, don’t tell you this story . And I think that this is really what’s so important for engineers to improve the experience and to improve their productivity, their performance. To reduce their pain points

[00:26:06] Dr. Margaret-Anne Storey: Actually, you reminded me of one good example when you talked about coverage We did this study that I mentioned that looked at the quality, how developers define quality and how managers defined quality. And we have this framework called truce, which is the timely delivery of robust features that delight users, support future collaboration and future evolution of the product. This definition of quality came from the developer and managers words that they gave us in the survey we did. Again, there are five dimensions ? Quality you can think of according to these different dimensions and test coverage is quite interesting because that would fall under robustness.

[00:26:48] That you have really good tests for the code that you’re delivering so that it will potentially catch some of the bugs that you have in the code that you push out today. But you might also have tests that don’t necessarily cover the code as it is today, but are there to support change in the future. So they allow you to make changes in the future. Obviously they cover the code still, but they’re more focused on not finding bugs today, but allowing somebody else on my team to make changes in the future. So how do you tease that apart? Understanding what it is that you’re looking at is really, really critical.

[00:27:24] Metrics, if you use them or measurements, any measurements, if you use them in a blind way, you’re going to be missing that nuance and they’re gonna get stale. And people will misuse them. And they’ll be abused and they’ll be gamified. So really bringing that nuance in is critical and addressing the misperception, I guess, that it takes a lot of time to bring that extra information and it doesn’t . We need to think about it and build a culture that it’s important to ask these questions along all of these different dimensions.

[00:27:56] Dr. Michaela Greiler: I think there also have to be role models. But I totally agree with everything that you said. I actually think we are at the end. I’m so happy that you were here and I definitely will invite you again.So thank you so much, Peggy, is there something that you think makes this conversation a little bit more rounded that is still missing from this productivity idea?

[00:28:17] Dr. Margaret-Anne Storey: I guess, just to sum up that if you’re thinking about developer productivity, also think about developer experience, and also think in terms of quality, like product quality and to really kind of think about those three independently and to think about if you make some changes or you’re making decisions, what are the dimensions of each of those three things? What are those dimensions that you need to think about? And if you’re working with other people. Find out what their assumptions are. If you’re talking in a team and somebody is talking about developer experience or developer productivity or software quality to say, hang on a second, what does that mean to you? Even if you do that, I think it’ll shift the conversation in the room quite a bit.

[00:29:03] Dr. Michaela Greiler: Unpacking their assumptions, this is so powerful. So thank you, Peggy. Thank you so much for joining my show.

[00:29:09] Dr. Margaret-Anne Storey: Thank you so much. It’s been great.

Do not measure developer productivity

In this special episode (50th) I’ll tell you a bit about the research on developer experience and developer productivity that kept me so busy this year. 


You will learn about:

  • some history of productivity focused thinking
  • new productivity research like the Space Framework that helps understand and measure developer productivity
  • the difference to the Developer Experience framework that I co-created
  • why (IMHO) metrics and a productivity-driven mindset can be problematic for your culture and long-term success
  • and what you should focus on instead.

Book your awesomecodereview.com workshop!

Subscribe on iTunes, Spotify, Google, Deezer, or via RSS.

Transcript: Do not measure developer productivity

[00:00:00] Michaela: Hello, and welcome to the software engineering podcast. I’m your host, Dr. McKayla, and today is a special episode. Today is the 50th episode. Can you imagine? Time flies! And as I announced last time, this one has a new format. It’s not an interview. No, no, no. There’s only me talking. There’s no other person joining me. Well, I want to tell you a little bit about the research that has kept me so busy over this year. I spent really hundreds of hours investigating what makes developers happy, productive, and satisfied with their work. I was especially looking at the concept of developer experience and what that is, well, that’s, that’s the content of this episode today.

What inspired me to talk about this is that I recently gave a keynote at a large internal conference at a large corporation to over 1,300 developers. And when I was preparing this talk, I thought, well, this could be actually an interesting topic also for the podcast. And yeah. It’s something that I know a lot about that I spent many years actually researching being in this area. And so, yeah, I wanted to tell you a little bit more about developer experience and code reviews, obviously, and productivity. And today’s episode is really about productivity and developer experience and you know, what those two concepts have in common and what makes them different. And so, yeah, well, let’s, let’s try it out. Cut me some slack, please, because as you know, I’m experiencing experimenting with the new format and I would also love your feedback. So I will link at the end. I will link a survey. You can always go to the survey and let me know what you think about it. Let’s start. And do you know why we start? We started in 1440

[00:02:00] that’s right. 1440. We are not stopping in 1940. He started in 14, 14. 1440. There was a very unsuccessful businessman that wanted to reveal a new and really astonishing invention. And he was hoping it’d with this new invention, you will be able to pay off all his death. In fact, he did not manage to do that, but what he revealed was really, really unique and very game changing. It changed a lot. Do you know what mean. It was the printing press and the, the guide revealed that boss called Johannes Gutenberg. And the printing press was a machinery that the loud to print books and other documents extremely efficiently. And what this means is that now books and documents could be manufactured in mass and it could also reach masses. So it was not only one book that was super expensive because it was hand printed or handwritten, but they could now print really a lot of books and became much cheaper. So everybody could actually buy one. And this invention lad to many, many. Changes, right? It led to widespread literacy and education, especially of the middle-class. And also change of thinking came with that suddenly a new thoughts, you know, became, could spread around very easily. And so this invention also played a key role in development like the Renaissance or reformation or age of enlightenment and the scientific revolution. And so it permanently altered the structure of society through mass communication. And this is why this printing revolution was so, so important. But also with this printing revolution and new thought was really made to work and that is productivity. So in comparison, if you had this printing press, you could produce up to

[00:04:00] 3,600 pages. But before the ad, you could only print 40 by hint printing or only a few by hand writing head copying. Right. So we really saw that suddenly we can produce much more efficiently. But this was obviously only the tip of the iceberg, right? There was the industrialization which came later starting mid 18th century and more and more production processes became driven by technology and by machine. And also the machines, like the printing press that was first hand operated now became operated by steam engines and electricity. And this led to another level of mass production and the thought of productivity became even more engraved in our minds. Right. But it also meant that a lot of professions became obsolete. Right. So we, we didn’t do. Everything from start to finish, like producing a shoe. There wasn’t like a Shoemaker that would know how to do that completely. But new and new professionals born at that was the factory worker. And because we were so focused on making the production process more effective and efficient, we realized that actually, if you do fewer staff, And, you know very, very focused task that this would lead to better results, more quality, but also really a higher production scale. And well this means that only even nowadays, right, a lot of those factory workers only have to do a few hand movements or tasks, so. With this, we could really reach a productivity level that we haven’t seen before, but you also noted that industrialization. Wasn’t where we stopped. Right? Mid of the 20th century, the digital revolution took place. And this is really important for us, right? Because now automation happened and that was enabled by computer.

[00:06:00] And once again, it’s replaced a lot of professions, but also it aided a lot of professions. Right. And so when computers took over work, that before hand was only done by humans, right. They could do it faster and with less errors than human. This technology was more, much more versatile than a technology before, right? It’s not one machinery that can do one task, but it’s a machinery that can do so many tasks in theory, perform endless operations. And so it’s very important to ask about, well, what, what can this technology enabling even at a Loveless already ask herself? Well, Can Kim computers even think, and she concluded a machinery is only as intelligent as we instructed. And while AI promises to shift us, we are not there yet. So we have to, we have to instruct machines. We have to instruct the computer to do it. And so this is all professional software engineers, developers, right? But there is also product managers are HITECH testers, UX designers, and many, many more, many more that really are needed to work with this versatile new technology and make it do what we wanted to. The problem is that productivity is still on the forefront of our minds. And it’s a goal that we have, right. But we face this dilemma that now these professions it’s not about doing so much as it’s about thinking, right? So the fuel of the productivity is our thinking, not so much our doing and now assessing and grasping what productivity even means becomes extremely difficult. Yeah, we try, we try we all know that, you know, if we want to measure productivity, we can do it by lines of code. We can also not, you know, measure our productivity by the number of meetings

[00:08:00] we attend. And the company, unfortunately, does not automatically gets more successful as we, you know, ride more. It doesn’t get more profitable with that. It’s not a guaranteed the recipe, but still we try, we try and we try and we also try to measure productivity because somehow we want to be accountable and we must be accountable organizations. I want us to be accountable. Our, our team wants us to be accountable. Right. And so we have to demonstrate our contribution and this thought of productivity. That’s very, very connected to activity and output is, is, is on the forefront of our minds. And so, yeah, we’re measuring, unfortunately, some companies, some teams are measuring lines of code or features now, although those kind of things, because we can easily access it. And I think there’s this heritage that we have from how we did it before. But it’s not as accurate anymore as it was before. and so yeah, I want to talk about that a little bit about productivity and how can we assess it and an interesting, very new. Approach to assessing this productivity problem comes from Nicole four screen from GitHub and Margaret and story from university of Victoria. And then there are a lot of Microsoft research faults like Chandra Mudela and Brian hook and Jenna Butler. But what did they do? Well, they created something called the space framework and the whole idea behind it is let’s look at productivity and let’s try to measure it. Right. And I think that the biggest takeaway from their work that they did is that. Well, we cannot measure productivity with one metric to one, you know, with one single dimension. And, and I think that’s very true and there’s also a really, really excellent book. I will link everything in the show note, which is called. I have it here next to me. It’s called

[00:10:00] rethinking productivity in software engineering. It’s not that new anymore. I think it’s 2019 or something. And it’s the, the editor where Tom Zimmerman as well, right. Also from involved in the space framework and Caitlin Spassky and there a lot of different researchers around the world actually write some essays. That are based on their research that they do about what does productivity mean? And here we have also to other people talk about well, there’s not a single metric, right? This is something that actually comes also from Kathleen’s Bardowski and, her colleagues at Google, right? Where you say, well, let’s actually not one similar metrics. You cannot compromise that. And if you look through the book, you will see that productivity can have so many different dimensions and people look at so many different things. What does productivity even mean? And but coming back to the space framework, I want to talk a little bit about this. And so this ad, well, there’s not one dimension that we can, you know, somehow. Measure productivity, right? So like lines of code, but there are different dimensions that you have to look at. And they, I think they are really interesting because it’s the first dimension it has to do. You have to measure something around satisfaction and wellbeing, right? So this is helpful filled our developers with their work, with the team. Oh, happy and healthy. Then you have to measure something about performance. And this is more in terms of outcome than output, right? Output would be well, the, the features, right? The number of features, for example, that we do, but outcome would be more, oh, how happy our customer. What’s the quality of our, softhearted VO developing. And does it make revenue, right? Something along this lines. And then you talk about activity, right? Activities very often, you know, in the center of our minds, when we have probably think about productivity and it would be for example, commits how many commits are happening or, you know, what are you

[00:12:00] doing? That very easily maybe can be measured through our systems that we have. Uh, Another dimension is communication on collaboration, right? How people in teams communicate and work with each other. And again, the idea is, again, how can we actually measure that? How can we put it into a metric? And the final thing is efficiency and flow, right? So do developers have minimal interruptions or delays for them? And if you look at the space framework, I’m going to link that as well. Then they have some example metrics, right? Because it’s all about metrics. And they give some examples, right? Like for satisfaction, you could have a developer satisfaction survey. So you know, a little bit about how people feel about it, about their work for performance. Well, he could have cultural view blossom for activity. You could have the number of code reviews completed, right? This is really just a activity metrics and then communication and collaboration would be well. You know, PR merge time is something that they came up with, right. And efficiency and flow would be the code from your timing. And while I think it’s really one of the. One of the most holistic views on productivity that I have seen so far. And I think it gives a good balance between those things. I still have the feeling that we are trying, you know, to measure things that are very often meaningless, right? Like for example you know, PR merged times. It has a meaning, but most of the time, I would say the meaning comes from really looking closely and qualitatively at each of those PR merge types. Right. So you can, if you, if you put up a dashboard. What will they say, right. What, what will happen to all of that? And then also, how can you combine those different metrics that you’re not having and get this, this holistic view of productivity? I think it’s very, very difficult

[00:14:00] understanding. And there’s also a nice essay, very short essay. So if you want to read it I will link it from Amy Cole as she’s a professor at the university of Washington. And she says why we should not measure for active. And she wants us to be really cautious because it can send the wrong incentives. Right. So when you’re measuring, if you start measuring, for example, cultural view, velocity, what will happen if people know it? Right. And if you didn’t know that this is actually something that you that you judge and is, is the changes in behavior. Is that really what you want? Do you want them to finish that pastor? But you know, are you lacking somewhere else? I think it’s a little bit tricky. And so I find the space framework extremely inspiring, but for my research, we took a little bit different stand on the whole, on the whole part. And it goes away from measuring productivity to more, to measuring developer experience because there’s a lot of research that shows. Productivity and satisfaction somehow hadn’t had, right. They somehow influence each other. They are entangled. The people that are more motivated, they are also performing better. It’s a lot of research around this. And so developer experience was something that we thought well, instead of, you know, looking at those metrics and really about productivity, let’s think about how do be puffy. Uh, About the work, how do they think about their work and how to do value, but right. Which is our definition of develop experience. So what are the encountered in their day-to-day work and how does this make them feel? And so we also looked at different dimensions that are. Influencing how a developer feels about their work. And I’m going to, to show you a couple are, tell you a couple of those dimensions, right? So we also thought about collaboration and culture. You can see there is a parallel to the space framework. We thought about developer flow and fulfillment. I

[00:16:00] would say to somehow goes into this efficiency and flow area. We thought about product management, which is not really there. Right. So it’s different. And development and release, which is really the tooling. And I think this is also very different. And so if you assume in a little bit to this developer experience framework, and I will link that as well. But I think that the basic concept is quite different than. We don’t want to have any metrics around that, but we really want deeply to understand what impacts, how good a person can do their work. A developer can do the work, right. And I define it as well. Developer experience is all about doing. Your best work joyfully. Right? So that you can do your best work and it it’s, it’s fun. And so I did really a lot of interviews with people and I think this fun aspect or that I’m proud of my work. I think this came up a lot. And, and so we want the, to understand what’s, what’s impacting people and we started culture collaborative ness, right? So supportiveness how connected to a field with my team? How, how safe do I feel? My work environment, all these are really, really important. There’s a lot about product management. Is that actually going very well? Right? Do we have clear goals, requirements? Can we work iteratively, right? Do we have reasonable deadlines? I mean, do you know how, how unreasonable deadlines, how do destroy moral. It’s crushing it’s soul crushing. I heard stories and, and with the soul crushing activities that we have, we definitely reduce our productivity. We reduce our performance, right? So we want good experiences for our developers. And so this developer, extreme Springbrook book is really all about. Understanding what makes for a good day, what makes for good work for joyful book that people can nicely work. And I think Dan, it’s all about how do we assess that? Right. So and it’s not metrics, we don’t, we don’t crawl data from guitar or get lab and then,

[00:18:00] you know, merge it and then it comes out the number. It’s really about understanding. I think this is so important. So this is really where I’m coming from. I always think it’s about understanding what’s really happening. So let’s say one of the things in the bucket of development and release would be the code based help. So really understanding how is our code base health, but not, you know, one metric that tells us, oh, we score a 71% out of a hundred. No, it’s about how do we think. Are we going into our code base and it’s really cumbersome. If we have to fix a bug or add a feature, we don’t want to touch this part of the code base. Well, this is really bad, right? And here we should something do something independent of some number that comes around. It’s about, you know, the flow, the flow would be how challenging was stimulating. Do I feel my, my work is right. Do I feel motivated to do it? How often am I interrupted? And it’s not, again, not that, you know, become that or that somebody observes, maybe the camera it’s really about how do I feel? Do I feel uninterrupted? Do I feel interrupted to have the ability to do. And in my courtroom workshops, I do the same, right. So when I work with, with teams, when I work with companies to improve the culture processes, I definitely look at data. Data is important, but it’s like a small thing. It’s it’s, you know, it’s, it’s, it’s tipping us off. You know, so let’s say if we are looking at our contribution, I definitely ask you, so how long are you code reviews? Taking which we could say, well, this is turnaround time or whatnot, but I don’t want you to build a dashboard and look at this number. I want you to get a feeling really, really understanding what’s happening here. Oh, you know, we feel, and maybe the best is to say, we feel it’s too low. Or we feel it’s just right. You know, this is maybe even more

[00:20:00] important than knowing it’s it’s a week or it’s a day. And obviously there’s evaluation in it. There’s judgment in it. Right. And then we can go back to the data and say, okay, you are not feeling so right. It feels, you know, we are a little bit too slow. It could be faster. How long is it taking on Everett? And then we look at particular things the longest one, how long are they taking? Right. And then we try to, let me try to really understand why are you taking so long? Are there specific code reviews that are taking it on? Do they have some certain characteristics? Right. And then because of that, this is really an investigation and an improvement mindset. And we can try to come up with solutions because if you have a dash. Then it’s one thing, right? It’s, it’s making those things faster. But maybe there are some cultural use that can’t be faster and they shouldn’t be fast that it should take long. Right. And it’s okay that they feel too long and, you know, so really looking at the particular uh, things. Yeah. So I also wanted in this new to format to make my, my podcasts a little bit shorter, I hope I gave you a little bit to think about and Maybe the main message here is that productivity metrics, right? They come, they come from a different age, I would say from a different time. And I think also in a different setting, they probably in an industry setting, they are still very relevant and they can be very. Important, but as knowledge workers. Right. But we think them do, or you know, where everything that we do should be thoughtful. It become very meaningless and I think they are really dangerous for organizations. I think it’s super atrophy that we, you know, sometimes take data to investigate, especially investigations around how are we doing a really good, but. I have never seen them really work very valid in dashboards, maybe very short term because we have one particular goal, but in general,

[00:22:00] it’s really about helping us to meet. Make decisions and make improvements, right? So the data should just help us to go in the right direction and help us to dig a little bit deeper, right. To know where to dig deeper, to really look what’s going on here. And I think the developer experience, I liked this contest concept a lot. I think that people that are happy that, that liked to work that can do their best work. Joyfully. If you can reach that then, you know, Trying to measure productivity or performance becomes really a little bit more obsolete, but I’m super happy to hear your thoughts on that. What do you think how are you assessing some of those things? Is there something, how do you keep yourself accountable, your team, your organization, and, yeah, and as I said, For the survey, I will link it. Maybe you, let me just know how you like this new format, this new episode, it’s a catastrophe, or was it good? A good start. Maybe some topics that you want me to talk about and that I was thinking about the sad question that I will end at there. Right. So three questions. It would be really, really grateful if you, if you let me know. And yeah, so that was it. And by.