In this Codecast, Tom gives an overview of test-driven development benefits specific to the Ruby language, offers a great live coding example of TDD in practice and shares useful criticisms, resources and fresh methods for new developers to test code in a proactive manner.
Key Theme » Tempered Testing
Tests aren’t free – they take time to write, they take time to run, they take time to maintain when they break. So you can write too many tests, definitely. There are a lot of different schools of thought on how to not over-test or what should be tested, what shouldn't. You kind of have to develop your own personal preferences there based on experience about: "should I test literally everything, or should I test nothing or, where’s the balance in between that’s good for this project that I’m working on at the moment?" – Tom Dalling
"...in Ruby, TDD and testing is basically just that – it’s a discipline so that you don’t cause bugs. It guards against the "sharpness" of the language so that you’re not..always introducing bugs every time you write code. You use the sharpness but you also have these checks in place to make sure you’re not doing the wrong thing."
"TDD is often used as a substitute for design architecture instead of actually thinking up the design architecture before you start writing code. This kind of stems from you’re doing a little bit at a time so you write a little test, write a little bit of code, little test, little bit of code. Over time that builds up into something - it’s sort of an evolved system not a design system and that evolved system might not have the best design."
Full Episode » Transcribed
Hello and welcome to the Viking Code School Codecast! My name is Erik Trautman, I’m the founder of Viking Code School and this is a weekly conversation with developers, software engineers, product managers, anyone from around the technology industry who is doing really cool stuff and who we can learn from about the role and the job of being a software engineer as many of you in the audience are either on your way to getting there, maybe already are there, or are looking in from the outside, maybe thinking about how do you connect the dots to make that happen.
With me today is Tom Dalling, who’s a full stack developer from Australia, who does OSX and iOS development, but he’s done all sorts of things across the stack. He’s done front end, he’s done back end, he’s worked with games and he’s actually written a book Programming for Beginners which you can find at ProgrammingForBeginnersBook.com. I believe it’s ProgrammingForBeginners, as well right, or is it just ProgrammingForBeginnersBook?
There we go. So he’s with us going to be talking specifically about a topic that a lot of you have heard about, many of you have learned about but almost none of you have a true affinity and appreciation for, because it takes a long time to get to – and that’s testing and test-driven development (TDD). This is something that Tom has worked a lot with, so Tom thank you for joining us. It’s our pleasure to have you. First off, welcome and thanks for joining us!
Thank you for having me, yes, it’s good to be here.
I guess to kick off - the first question that I like to ask just to get a sense of who you are is: Tom, who are you and how did you get into development? How did you get to where you are today?
I probably started writing code some point in high school, I don’t remember exactly what year, but I would have been about 14 or 15 (ish). My dad was doing an IT degree at University at the time, and he had Visual Basic installed on a computer at home so I would play around with that. Wrote some, just like annoying things usually just to share with my friends at school and that’s sort of work encouraged me to actually pursue it at University. So once I left high school I did an IT and Business degree at a smaller university in Australia. Since then, I’ve just been a professional programmer. The first job I got out of university was doing desktop OSX stuff, before that just like part-time stuff during university hours doing PHP web development, back in the "bad days" of PHP.
Does that mean that you work in PHP now? (Laughs)
I don’t actually know what the PHP days are like at the moment, but I hear it’s a lot better but back when I was doing it, it wasn’t so fun. (Laughs) Yes, so I’ve always been interested in the website and I’ve been using Ruby since university, not always professionally but for the last two to three years I’ve picked it back up again. I’ve been doing web stuff, like web with Ruby on the back end, and mobile apps for the front end. That’s basically me, up to this point.
I guess before we dive into the testing stuff, what was it that ended up getting you into Ruby now from the previous environments you’ve been working in? You said Visual Basic and working in OSX is fairly different.
Visual Basic, I started with that and even back in 2003 (ish) when I was using it, everybody was saying it was a terrible language, so I switched to C – pretty quickly. It was like C++ and then in university we did Java and C++, but that’s where I started doing Ruby. I had Visual Basic, C, C++, maybe dabbled with Python and PHP experience at the point that I picked up Ruby.
I have been using it for a long time, even before I had ever even seen Objective C I already knew Ruby, but coming back to it after quite a few years, only really doing Objective C development was interesting. I liked the direction that the language had taken. There is certainly a lot more stuff, like Bundler. Bundler was not really a thing when I started, and it’s just fantastic now, I love Bundler. The language is quite a bit faster as well and I was quite pleasantly surprised to see how it had evolved over the years. It wasn’t too hard to pick up again, I sort of knew what I was doing already.
One of our focuses for today is around testing, and specifically testing in Ruby. Had you been doing a lot of testing prior to getting back into the web?
Not a whole lot. We did have some level of testing in the OSX app-land, but interfaces like graphical user interfaces (GUI) tend to be pretty terrible to test, like very difficult to test. So the testing was patchy at best, probably, around that. But coming back to Ruby – to go back to what you were asking before.
One of the things I did notice coming back is going from working predominantly in a statically-tight language moving to a completely dynamic type language, you write more bugs. So I’d start writing and more bugs would pop up. It was sort of like I relearned why testing is important, (laughs) at that point. After a little bit of that you can start writing a lot more tests. Yeah, I did have to pick up the testing when I switched languages.
That’s actually a really good segue into a logical next question which is: why is testing important? Why should developers who are learning to become software engineers focus a lot of their energy on testing?
Why is testing important?
When you write stuff – it’s easy to write code, right? It’s easy just to bang out stuff and be refreshing a browser role, clicking around in an interface to see if it works. But when you send that out to thousands of clients, they’re going to use it in a lot of different ways that you wouldn’t naturally have tested – just clicking around and things like that. So you end up with I jokingly refer to as "customer-driven development" where you send out this buggy thing to the customers, they complain back to you and then you fix it and send them out a new one – which is not a good way to develop software. You’re using your company’s reputation as a way to get quality software, which is not what you want to be trading off. Ideally, you want it to just be working all the time when you send it out and you really need automated tests to make that happen.
How is the environment for testing, you mentioned Bundler coming in, I think that’s a pretty good milestone for testing frameworks and things like that, but how have you seen the evolution of the tools at your disposal for doing testing? How has that occurred since you began adopting that as a necessary portion of your own workflow?
I can’t actually remember very well what I was doing testing wise, back when I originally did Ruby so I can’t really comment on how that has worked. But I have definitely kept up with reading news and things about how Ruby on Rails have gone. It feels to me like Ruby really has a culture of testing and they’ve certainly been pushing the boundaries of what is good testing and testing frameworks, and things like that and that sort of spread to other languages and other frameworks.
Ruby as "compiler-free," dynamic option for testing.
I think Ruby is especially focused on testing as opposed to other languages, but the dynamic nature of Ruby is a large reason for the emphasis on testing because you don’t have a compiler to check your code. You can just write in, fix the bugs and run it and it might run fine most of the time – until you do this specific thing and then it crashes. So you need testing to pick that up, whereas a compiler might have picked that up for a different language.
Alright, so one of the analogies that you’ve used over time is the "sharp knife", do you mind just describing what exactly that is and how does that fit into the idea of testing?
Sure. That came from actually DHH (David Heinemeier Hansson), the original creator of Rails. He said that they "provide sharp knives." So Rails gives you sharp knives and if you don’t use them responsibly that’s not their fault kind of thing. The analogy is that, if you look at professional chefs they don’t use blunt knives even though maybe that’s safer. They might not cut themselves as much if they did use blunt knives.
Ruby as a "sharp" development language tool.
The reason why they use sharp knives is because it makes their job a lot faster, they can just get things done way faster with a sharp knife than with a blunt knife. It’s easier, they don’t have to put as much effort into cutting things, and it’s probably also a bit more fun. It’s no fun hacking your way through something, it’s nice if it’s just slicing through air. It’s more - I don’t know what the word it, it’s just more ergonomical, it just feels better.
Ruby is kind of the same way, in that it lets you do basically anything. It doesn’t try to stop you from doing bad things and you can certainly write terrible messes in Ruby, but on the other hand – its extreme flexibility is also good. It lets you write things more quickly. It can be fun and it’s that trade off between having fun and writing things, getting things finished quickly versus like how many bugs you are introducing. That’s the "sharp knife" analogy.
So then to continue that analogy, where does testing then come into that, in test-driven development in particular?
TDD in Ruby resolve bugs efficiently.
To go back to the chef’s, they have these really, really dangerous weapons that they’re using every day. What you’ll probably find if you speak to professional chefs is that they all have very rigorous rules and discipline around how you use a knife safely. So you curl your fingers back, you never cut towards your hand, you always use a chopping board things like that. And in Ruby, TDD and testing is basically just that – it’s a discipline so that you don’t cause bugs. It sort of guards against the "sharpness" of the language so that you’re not just always doing - you're not just always introducing bugs every time you write code. You use the sharpness but you also have these checks in place to make sure you’re not doing the wrong thing.
So then moving forward what exactly is test-driven development and how does that work in Ruby?
What is Test-Driven Development?
TDD is actually pretty simple at its core. It’s a few steps:
Firstly, you write a test before you write any other code. That test should fail, obviously, because there’s nothing written to implement it.
Then you go and implement your code that meets the test.
Then you run the test again, and the test should pass – and you just keep repeating that.
The emphasis is you have to write the code for the test before you write the code for the implementation. That’s the core of it. There is a ton of more complicated stuff to it, but at the core [those are] the three things: the red failing test, green passing test and refactoring after that.
Why is that useful?
There are a few reasons why writing the test - TDD’s usually compare to other types of testing, and the difference is that in TDD you write the test before the code. In other types of testing, you write your code and then you put in the test afterwards to make sure that the code works. That’s the main difference – whether you’re testing first or last.
The benefits of testing first are that when you do actually go to implement your code, it tends to be a testable code – because you’ve already written the test. You don’t go off and write this big bowl of spaghetti and then try to write the test afterwards, and go: "this is actually terrible, I can’t test this very well." It encourages very testable code that’s sort of broken apart and decoupled.
TDD provides a motivating & immediate feedback loop.
It’s also, I would argue and a lot of people argue, it’s more fun to write the test first. So you get this feedback loop where you’re doing a little bit of work and you get this reward – it shows up as a passed test, a little bit more work another reward, a little bit more work another reward. So it sort of plays into the motivation of your day as well. Because you’re taking little steps it also helps you to break down larger problems. You’re just doing a little bit at a time.
TDD helps new developers manage building more effectively.
I know that beginners can often look at a huge task they’re given to implement and just feel like: "how am I ever going to - this just seems massive, how do I do this?" I think TDD helps in that regard because you’re just taking a little bit at a time. You’re making just this one test pass and then another test and then another test. It just helps you to get going with larger tasks.
Executing TDD prevents laziness in software development!
Another benefit is that when you write the test after your implementation code you tend not to write the test. We’re all under time pressure. We have other things that we need to do, more features to implement things like that. You’ll write a feature and you’ll get to the end and go: "okay, now it’s time to write the test but it’s five o’clock and I’m going home now, or but I have this huge list of features that I have to do." So the tests tend to not get written when you write them afterwards, whereas with TDD the tests are just there – it helps you to write the tests instead of forgetting the tests.
Knowing TDD boosts your hireability as a Ruby developer.
The last point that I’d make about TDD is that it helps you to get a job in Ruby. Because Ruby (laughs) has such a culture of testing and specifically it’s very TDD focused, if you want a job in Ruby the person hiring you is definitely going to be looking to see if you have experience in TDD. So it’s worth learning just for that reason, really. That’s what I would say are the main benefits that TDD has over just normal testing.
And so with Ruby in particular, there are a number of different testing frameworks, but how does testing actually kind of work in Ruby? Like if I really haven't coded too much, let’s say I know Ruby but I haven’t really explored testing yet. How does it work and does the testing framework sort of bolt in?
I’ll give a quick explanation and then we can do like a live coding example if that’s cool. Basically let’s say you’re starting something completely new. You open up a test file and you write a class that has certain methods in it, and those methods will be automatically run by the test framework. So you start a new field called "foo tests", and I write in a test for "foo" and then I run it and it will say this crashed it didn’t work. Then I open up the actual thing that I’m writing, "foo.rb", and I start writing it to make it pass the test then looking back between the test and the implementation as you’re going.
That’s a quick overview, shall I jump into writing some code, at the moment?
We love code, let’s do it! Again, while you’re setting that up just a note for everyone else. You’re welcomed to interject and ask questions at any time.
Live coding example on how to run a test.
Okay, so I’m hoping that everyone can see my terminal. I’m going to start completely from scratch, I’m just going to open up VIM and the example that I have sort of prepared for today is a binary converter. So it’s going to take in a string of zeros and ones and it’s going to tell you what number is that.
It’s a pretty simple function. It’s already built into Ruby but we’re going to pretend that it is not built into Ruby just for the sake of showing things. So I’ll get started by putting up a file called "binarytest" in Ruby.
2 kinds of Ruby TDD frameworks.
This is going to be where the tests live and I’m going to be using a test framework called MiniTest ... which is definitely the default framework in Rails. Rspec is the other popular one – but I think MiniTest is just a bit simpler.I’m going to require this file doesn’t exist yet. It’s going to be called binary.rb.
Make the test pass, MiniTest-Test...this is the actual test here. Then we write the code that we sort of wished we had. So in this case, I’m just going to make up with the end phase will be. I happen to know (10101) is 21 in binary, and so this is the first test. At the top we require the testing library, then we require where the implementation will be but it doesn’t actually exist yet. We make a class that’s a subclass of MiniTest-Test, make a method that starts with Test (underscore), anything with Test (underscore) will automatically be run by MiniTest, and then this insertion here search is the actual test itself. It says “make sure the result of binary.parts (10101) is equal to 21”. If that’s true – test passed, if that’s false – test fail. I’m going to run this file right now and it gives us a load error – "required relative load file binary" – which makes sense because the file doesn’t exist.
The next step is to actually make the implementation, ...with binary.rb and I’m going to just save that file and run the test again. It still failed but with a different error this time – "uninitialized constant binaryæ – so there is no thing called binary, so that also makes sense. I’m going to make that, leave it, run the test again, and find method passed for binary module. That makes sense ... (inaudible) wrong number of arguments given.
This is the first time it hasn’t crashed, it’s actually failing because the values are wrong, so it expected 21 and it actually returned "nil", which Ruby just returns "nil" by default if there is nothing in the method. So we will put 21 here on the test, and that’s the first passing test. It doesn’t actually pass the binary string, but it does pass the test, and that’s sort of important for reasons we will get to a bit later.
But now that we have just a passing test, jump back into the test file and we try a new test. So I’m going to try passing 22 which is (10110) and we run this test. I have one passing and one failing test now, expected 22 and actually got 21 because it’s always returning 21. At this point, I’m going to actually implement the algorithm that will turn a binary string into the parsing (inaudible) integer, so I just know this all by heart. Reverse the string ... get all the characters ... need to know which index the characters are at ... and then map them. Every digit in a binary string runs as a "power of 2" number, so I’m going to turn each digit into its "power of 2" number here.
It’s not super important, an algorithm, it’s just mainly to show it will pass the test in the end. After each digit is turned into its power of 2, just add them all together and that should theoretically be correct result. Run the test again and I have two passing tests, which says to me that this algorithm is actually correct.
That’s pretty good for a first try.
A core TDD operative: "red-green-refactor."
(Laughs) I think it’s my third one, I practiced it twice yesterday just to make sure it was going to work. (Laughs) The idea with TDD you often hear the phrase, "red-green-refactor", and that’s just sort of what I showed then.
Red means you have a test that fails and it’s important to have a failing test before you have a passing test. Then green is you write the implementation to make the test pass, and then we’ll get to refactor in a second.
I want to show you why that red step is important. I’m going to break a new test, and I want to test that when I give an invalid binary string it’s going to raise an exception – so it’s going to detect that something is wrong.
In this case, when you assert raises, this will check that an exception actually happened, binary.pass "hello". You should never get "hello" in binary pass, it just doesn’t make any sense. I will run this test and I’m going to expect that it’s going to fail - and yet it doesn’t. I have three passing tests.
This is the reason why the red step is important – it’s because this is failing, but not for the reason that I want it to fail. So this test is pretty much useless, it’s not testing anything. So to fix this up, to make it fail before it’s actually implemented, I’m going to put an extra assertion on this and make sure it has the message that I expect.
I want this to say "invalid binary string" in the message exception. If I run this now it does fail, and it says that it should have been "invalid binary string" but the error message is "invalid value for integer" and the letter O. Way down here in the implementation where I entered a digit, that’s what’s throwing the exception, and what I actually want at the top, is I want to make sure that the binary string is correct before I even do anything. Now that I have the failing test I can write that code here. I will do that with a regular expression, it says it has to have one or more zeros or ones, and if it doesn’t match that, it will raise an argument error...(inaudible) "invalid binary string".
Now it’s passing again. So the fact that the test didn’t pass the first time indicated there was something wrong with the test and now we fixed that and now we have exception raised on how it’s supposed to be raised.
The last step is the refactor step. With these three tests in place I can change how this is implemented now and I should be able to tell if it’s working or not. If I’m looking at this and think: "that looks kinda inefficient, I could probably do that in a more efficient way." I can just delete out the implementation and start again.
I know that I can do this better using bit shift operations, which I’m going to do now in each character. Increase the result by the power of two (inaudible) ... and all the tests will pass. I have confidence that even though I’ve swapped out the implementation for something else, it’s still doing what it’s supposed to do – instead of maybe it works, sometimes, maybe it doesn’t. I’m actually kind of a little bit surprised that worked first time so I might change a little thing and run it again so it’s failing.
Yeah (laughs). If something passes the first time you should always try and break it.
Yeah, yeah. Definitely, and I’ve seen other people do this as well. It’s not just me monkeying around it’s a legitimate thing that you do. If you pass too easily that is probably an indication, or it could be an indication that something is wrong with the test. So definitely break the test on purpose and make sure that it’s working.
This is the last step. The refactor step is pretty important because if you’re just going red green, red green, red green, you can end up with a mess still. It’s a mess that passes all the tests and it works, but maybe it’s not written the best way that it could. So it’s important to go back after all your tests are green, make the code nicer. I’m even going to take out this regular expression here, because I want it to be a bit more descriptive about what it is ... "valid binary equals that". I’ll test again and it’s all good. That’s the nitty-gritty of how TDD works, or at least a small problem.
Cool, does anybody have any questions about the code in particular before we move back into the conceptuals?
So I guess one of the most common questions that I hear from students is - there’s the "how do I test?" Which is ultimately at the end of the day an acquired skill that comes out of mechanics.
But then there’s the "what should I test?" We’ve heard the idea of happy pass, bad pass, or sad pass and bad pass and things like that. But what exactly is it? Let’s say you’re building one of your games or something like that. How do you sort of approach this idea of A) where do you start and B) what should you be writing tests for?
Tips for knowing what to test.
There are a lot of different opinions on what you should and shouldn’t test and there are also a lot of rule of thumbs that work a lot of the time that maybe don’t work in all situations. I would say that the easiest thing to test are pure functions. In this case, this binary passing is the pure function. You have an input and it gives you an output. Whatever happens under the hood could be extremely complicated, it could be simple. Those are just really easy to test and you probably should be testing those, no question. They’re testable, so why not?
Other things – like games in particular – they tend to be actually very difficult to test, a lot of the stuff. I have seen Valve, a pretty popular game development company – I have seen them do testing where they’re porting a game from Windows to say Linux, and they run both versions simultaneously. They’ll have a Windows machine, a Linux machine and they’ll be running both games with just one keyboard and mouse, so they can see both games. But they also have a window in the middle that’s the difference between the game on the left and the game on the right.
It will show up any tiny little - the color is wrong, is this object in the wrong position, things like that will show up to the developer. That is sort of an extreme end of testing where you’re trying to get to pixel-perfect rendering, but in general interface stuff like that, like the game, tend to be really, really difficult to test. You’ll probably find in the games industry that they don’t care about testing that stuff at all. They usually have - not automated tests, anyway – they’ll have QA people who will play the game non-stop trying to find these bugs, as opposed to writing automated tests like we just did then.
One factor about choosing about what to test is: how easy is it to test? If it’s easy then definitely do it. If it’s difficult - you have to weigh out whether the cost of testing is going to be worth it to you.
What else do you want to talk about in terms of what to test?
So, one set of the challenges it seems everybody has (inaudible) is command lines games like "Tower of Hanoi" or things like that. It’s kind of devious because, if you’ve built it in the proper object-oriented fashion, you have classes and methods that you can test using unit testing style. Meanwhile you’ve got this loop – and the core loop just throws a lot of people off. Like, how do I even start testing something that has this loop, like until the game is over keep asking players for input and things like that?
Do you have any heuristics that you use for sort of just getting the ball rolling, like that first bit of momentum in TDD?
Some methods to test difficult programming functions.
Sort of - yes and no. That sort of game loop, that is an example of something that is quite difficult to test. It’s not impossible to test – but it’s probably the harder part. Depending on what I’m making, I might not even test that. I would try to take the functionality in that loop and break it out into separate classes that are easy to test.
You want to minimize the part that you can’t test and maximize the parts that you can. If you have something that prints the output of your "Tower of Hanoi" game, you want to make that into something separate called like "output formatter", probably, and you can give it a game and you can test what it gives you back. You can make sure that’s making the string that you want, then you introduce that back into the game loop which is maybe untested. Yeah, I would probably in that case, I’d be running the game to make sure that part’s working fine, plus trying to take as much out of the untested parts as possible into testable processes.
In terms of like a web app, which is probably what most people working with Ruby will be working on, you sort of end up doing integration tests. Which are - they’re not testing one little bit of functionality they’re testing the entire system after it’s booted up.
You’ll be requesting a page and it will run through your entire app, through the controller's down to the model down to the database, come back from the database all the way up. You’ll get HTML rendered as a response. Then you’re checking the HTML, saying: "did it include the things I thought it would include?"
So that’s one way to test stuff that’s not easy to test. You just test the entire system.
To go back to the game analogy. "Triple A" titles, like big expensive games, will often record every single action that the player makes into sort of like something they can play back again. They will be testing the game, clicking, then the game will crash – and they have no idea why it crashed but they have this recording of everything that the player pressed. So they send that off to the developer. The developer replays that sequence of input and they can re-trigger the crash again.
That’s basically an integration test. It’s running an entire game, simulated game just to get to the point of the crash. I’m not sure exactly where I’m going with this, but I guess what I’m trying to say is: if you have something that’s very difficult to test you sort of have to capture the input. Like what’s the user actually doing? And that can be difficult depending on your domain.
For a web app you’re capturing a sequence of requests to your app.
For a game you’re capturing a sequence of keypresses and mouse moves.
To go back to your "Tower of Hanoi" example. You could theoretically do the same thing. You could capture everything they press on the keyboard and you can replay that to make sure that it works. Your test would be a sequence of actions that the user takes and you run that through your whole program and you sort of make sure it doesn’t crash, and maybe test the output to see how it works. Again that’s complicated – it’s a lot more complicated than a simple unit test. So you have to weigh out the cost of doing that versus not doing it.
Hopefully, that made sense.
Yeah, thank you. I definitely have some places I think we can take this a little bit, but I’d really like to open things up to anybody out there who has questions, so please ask.
I had a quick question. If you were adding tests for an app, a web app, that didn't have tests before – you’re trying to add test to legacy code, or something.
Would the first natural step be to start at the low level, like the really easy tests, the unit tests and go up to the top with integration tests to kind of refactor the code and make sure your web app runs as intended? Or, is it better to go from the other way around kind of approach, to kind of refactor and make sure that web app runs optimally?
It kind of depends on what you’re doing with the web app. If you’re changing existing functionality you probably actually want to start with an integration test because you’re not going to be able to test the small parts very well, but you do want to make sure that when you make a change you’re not breaking things. So you probably want to write a test that tests the current functionality. Then you make your change and then you run the test again to make sure that nothing broke.
Coming to a large project without integration tests are pretty important, especially because you don't know the code base, you’ll probably break a lot of things as you’re starting. You can pick smaller things if you are actually working on a smaller subsystem, then making unit tests down at that level is totally fine.
Then again: you don’t know where that unit’s used in the entirety of the app, so it’s gets complicated again. Whether to start with integration or from the lower level I think integration gives you better confidence that you haven’t broken anything, but if you’re trying to increase the test coverage of your app and you don’t know where to start it’s totally fine to write the lower level tests. You want everything to be tested eventually anyway, there’s nothing wrong with writing those tests - if that helps.
Cool, yeah thank you.
Awesome, any other questions?
I have a question - am I audible? Good, my name is John Kosmicki, by the way. It’s nice to meet you.
Just bringing back parts of the curriculum that we dwelled on quite a while ago, and forgive me I may not remember these things quite as well as during the time when we were explicitly working on them. If I remember correctly my biggest issue was, I really didn’t understand the test as much as the fact I didn’t understand the problem. Or I didn’t understand the application, didn’t understand the function of what I was trying to build.
So, I would have this idea of the "red-green-refactor", and that’s cool it works just like you demo’d it. Everything is going to be peachy and so on. Then when I do it, it always ends up something like ... I write the test or I write the function then I realize I really don’t want this function. I really want this other function instead that maybe tweaks something a little different or returns a little bit different value.
And then, the test that I had really is not that important except for the fact that now I know I have to write this other function instead. And it really doesn’t quite fit into the "red-green-refactor" methodology.
Am I doing something wrong? Or maybe I’m just forgetting kind of the way that turned about. But that always seemed to be kind - it’s like you want to use the test to validate your code but if you don’t understand what you’re trying to do well enough then your tests are bad, too. At least, not meaning to be too negative about it, but that was sort of the problem that I kind of ran into.
Yes definitely, I know exactly what you’re talking about and I have a resource for you that you’ll probably find very helpful.
I don’t know if this goes in the show notes or whatever but it’s a series of videos by Justin Searls called My Favorite Way to TDD. We’ll probably link to this afterwards, but basically that is a problem with classical TDD.
You have to write the test before the code, but what if you don’t know what the code does exactly. That’s a pretty common problem. So in the example that I just gave: "why did I use a module instead of a class?" "How did I know how to use the regular expression, how did I know to raise an argument error?" That sort of thing. There is nothing really about TDD that informed that, that’s just my experience speaking there. I knew how to design it before I wrote the test, basically.
There is no good answer to that. There’s nothing that’s going to fix all those problems overnight. It really does come down to studying sort of architecture and design and coming up with your own sensibilities about what’s good design and bad design. The thing that I linked to by Justin Searls, he talks about classical TDD as not really talking about design and he has his own sort of style that he called "Discovery TDD" where you actually start at the top level.
So you start writing code not knowing exactly what it does but it ends up working out. So definitely go and watch those videos. Give it a try for yourself it may help you with that specific problem. But yeah, I know exactly what you’re talking about and it is a really, really common thing. I hope that helps.
Yeah that’s perfect, I don’t want to take this offline but I take it this is going to chat or the YouTube chat or something?
We’ll put it in the transcription.
Justin Searls by the way, he is sort of - I really like his approach to TDD. He has very pragmatic sort of no-nonsense, a bunch of videos. You can find him at TestDouble.com and he goes to various conferences and things. I think his perspective on TDD is very good and I would definitely recommend checking out all his stuff.
Cool, any other questions?
Alright, well I think kind of going over the hump from practical TDD to conceptual or beyond, I think there have been pros to TDD but there are also some criticisms. Do you mind talking about some of the downsides to TDD which is obviously not a bad transition from that last question, and some of the iterations and variations on it?
Criticisms of TDD.
Sure. So in the Ruby community TDD has almost taken on a religious aspect in that it’s everywhere and some people even see it as it’s the only way to do things and it’s always right and it never fails, which I don’t really like.
TDD is just one tool in your toolbelt, testing first is good for certain things and it’s okay for others and maybe even bad for other things, so to talk about a couple of the criticisms. The first thing that I came up with actually just what we talked about.
1. TDD is often used as a substitute for design architecture instead of actually thinking up the design architecture before you start writing code.
This kind of stems from you’re doing a little bit at a time so you write a little test, write a little bit of code, little test, little bit of code. Over time that builds up into something - it’s sort of an evolved system not a design system and that evolved system might not have the best design. It might not have a design that’s as good as if you have set down and worked out what it was supposed to do before you started writing the tests.
Classical TDD does encourage you to not really think about that, it’s do a little bit, do a little bit, do a little bit.
Another thing that I sort of noticed is that in the beginning when you start testing before you write the code it’s sort of a revelation. Like the experience is really good for beginner to intermediate programmers if they’ve never done it before. It actually makes the code that you write better, you are just more motivated to work, it’s really good. After you’ve done it for a few years and you’re used to it it’s comfortable, you can actually start testing first and you basically write the same code anyway.
2. The advantage is in sort of what you learn from it in the beginning. Then once you’ve got that advantage it’s not useless – but it’s importance is certainly diminished.
I was watching a conference talk from one of the original signers of the Agile Manifesto, and he said up on stage that he was a TDD follower and now he just doesn’t. He noticed that there was no real decline in the quality of code he wrote when he started writing tests after as opposed to before. I think that really does come down to once you’ve learnt the lessons, testing before isn’t as important.
3. It can also sort of lead to over-testing, where you try to test literally every single thing in the system from multiple angles.
Tests aren’t free so they take time to write, they take time to run, they take time to maintain when they break. So you can write too many tests, definitely. There are a lot of different schools of thought on how to not over-test or what should be tested what shouldn't. You kind of have to develop your own personal preferences there based on experience about: "should I test literally everything or should I test nothing or where’s the balance in between that’s good for this project that I’m working on at the moment?"
4. The "red-green-refactor" rules – just to move onto another criticism, the rules red green refactor that’s the core and that’s a nice, pithy way to explain it.
But TDD is actually really, really complicated. There are so many different opinions and rules - just to go over a couple, some people say every single test should follow the (inaudible) which is like a way of writing your tests and comping code together. You should only have one assertion per test. You should only write the absolute minimum amount of code to pass the test.
You notice when I did my live example I made the function just return 21 because I knew that’s what the test wanted. That’s actually something, it’s a rule that people follow and they think it’s good - well if you could just return 21 why don’t you return 21. If that fits the test then that means the tests aren’t good. So it forces you to go and write another test that makes it break.
5. We haven’t even touched mock stubs, double spies, these are all - they are sort of "fake" objects that you test against, instead of testing against "real" objects.
Let’s say you’re testing something that goes in and makes a tweet on Twitter. You probably don’t want to actually go and make a tweet on Twitter every time you’re running a test, so you make a fake object, a test double that pretends to be Twitter. Those again are really, really complicated and everyone uses them slightly differently. There’s spiking which is I don’t know exactly what I’m writing yet so I’m not going to test first – I’m just going to write a bunch of code, see what works then I’ll probably scrap it and go back into TDD again.
There are rules about when can I spike? Is this actually spiking or am I just writing cowboy code with no tests? It’s complicated. If you just tell someone yes, "red-green-refactor, it’s easy!" It doesn’t cover any of that stuff, you’ll spend years refining how you do testing. Some of it’s going to be TDD some of it’s not going to be TDD, but I wouldn’t get too wrapped up in the "have to test everything first."" Just try to get as much influence from people as possible and try to just keep the things that work and for other things that don’t work. That’s just something that takes time.
Alright. Well I think we probably have time for one more question here, and I think we are probably good to go. We have one in chat which is which testing framework do you find most effective?
I like MiniTest and I also like Rspec. What I ideally want is a hybrid of the two. (Laughs.) So Rspec is definitely the more complicated framework and it looks nicer, but I don’t necessarily want my testing framework to be ultra complicated. On the other end, you have MiniTest which is really simple but it’s missing things that Rspec has and it’s not as nice to use. So they both have their merits, and I would say what I prefer and what I have actually been doing recently is I start with MiniTest. Then I build in the parts of Rspec that I like and I sort of use a hybrid approach.
Okay. Well I think that’s all we have time for tonight, or this morning if you’re in Australia. I’d like to say first of all Thank you Tom for joining us! Thank you everyone else who has been here. Tom, how can they find you on the web and kind of get in touch with some of your projects, if they want to?
My main blog is TomDalling.com. I have a short book for people who have never programmed before, so this is people who don’t know what a variable is, sort of level, and that’s ProgrammingForBeginnersBook.com. More recently I have started writing about RubyPigeon.com, so expect to see articles coming out maybe a little bit more frequently there.
Alright, well thank you so much for joining us, everyone here I hope you all will join us again next week. This is the Viking Codecast, Viking Code School signing off. Thanks a lot!
Thanks for having me.
Contacting Tom + Resource Links
My Favorite Way to TDD by Justin Searls