Git Product home page Git Product logo

ebook's Introduction

A Friendly Introduction to Software Testing

Bill Laboon

for AKS and CKN

Compiling this e-book

This textbook is comprised of a series of Markdown files, compiled into PDF format via PDF\LaTeX. Required dependencies, available through most package managers, include:

With the above packages installed and added to your PATH, simply run make to compile this document. The output is a PDF file named software-testing-laboon-ebook.pdf.

ebook's People

Contributors

benjaminmuscato avatar blaken avatar brandonhang avatar carols10cents avatar chris-slade avatar glesica avatar laboon avatar ritwikgupta avatar robbiemckinstry avatar rossacheson avatar sdzivanovich avatar shepmaster avatar timparenti avatar willengler avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ebook's Issues

TODO example in Section 13.11 (file 40).

The end of Section 13.11 ("Using Reflection to Test Private Methods in Java") contains the code:

// TODO: TEST HERE

This example should either be completed or removed. See text/40_unit_testing.md.

Downsides to hooks: there's part of the real interface you're not testing

In section 16.3 Provide a Scriptable Interface, you list some of the downsides to adding test hooks to your system:

It’s a security risk, for one thing—if someone discovers how to access the test hooks, they may be able to determine hidden information, overwrite data, or perform other malicious actions (or simply be curious and make a mistake). Adding a scriptable interface will require additional complexity in the program, as well as additional program length and size. The interface may be a drag on performance. Finally, time spent working on the interface means less time spent writing other features of the software, or improving its quality.

I feel like some of this gets close to, but doesn't come out and say, that you're now deliberately bypassing the way real users will interact with the system and you may miss defects that happen all the time in the real interface but never happen in the scriptable interface.

Chapter 13 (file 40) is much longer than the others.

Chapter 13 ("Unit Testing") is far longer than any of the others. The number of subsections involved causes slight formatting issues in the Table of Contents.

Consider splitting text/40_unit_testing.md into two or more chapters.

Unclear sentence in Injection Errors section

Apologies for the rash of submissions, I was on a plane so had to save them up :)

In section 7.1 in #9 Injection Errors, I'm not sure what you mean by:

It could be a language that a different system used but the data is passed in.

The data is passed in where? Which way are we going?

Absurdly long subsection heading

There is a bit of irony in the fact that Subsection 14.7.3, "Unit Tests Should Have Few Assertions (Preferably Only One) And Simple, Direct Names" is one of the only ones without a simple and direct name. It doesn't even fit on one line in the PDF (see page 119), and although pdflatex handles this fairly gracefully, it's still non-ideal.

From someone who, just five chapters earlier in 9.3.1, pokes fun at long titles such as The Egg, Or The Memoirs Of Gregory Giddy, Esq: With The Lucubrations Of Messrs. Francis Flimsy, Frederick Florid, And Ben Bombast. To Which Are Added, The Private Opinions Of Patty Pout, Lucy Luscious, And Priscilla Positive. Also The Memoirs Of A Right Honourable Puppy. Conceived By A Celebrated Hen, And Laid Before The Public By A Famous Cock-Feeder, I am disappoint.


Other less absurd, but still too-long-to-fit-on-one-line headings include:

  • Page 21: 4.1 Equivalence Classes and Expected versus Observed Behavior
  • Page 34: 5.2 Functional versus Non-Functional Requirements (Quality Attributes)
  • Page 91: 12.1 Benefits and Drawbacks of Manual and Automated Testing
  • Page 180: 20.5 General Guidelines When Developing a Security Testing Plan

Missing benefit of unit tests-- speed

In 13.1, in my opinion one of the biggest benefits of unit tests is missing from the list-- how friggin fast they are! You usually don't have to spend a lot of time setting up a lot of interconnected components, or opening browsers, or querying databases, or waiting on the network. The faster your tests run, the more feedback coverage you can get out of the fixed amount of time you have before you get bored waiting for your tests to finish.

What about comparing test case runs over time? Is that a thing?

It's kind of a thing I've wanted Jenkins to do but haven't gotten it to do.

What I mean is: say we have 3 test cases and we've done 4 test runs in the last week--

Test Run 1 Test Run 2 Test Run 3 Test Run 4
Test 1 F P P P
Test 2 P F P F
Test 3 P P B B

F = failed, P = passed, B = blocked

So we could see that Test 1 was broken but now it's fixed, Test 2 is flaky and either the test or the functionality isn't consistently reliable, and something happened to block Test 3 two runs ago and is probably still going on.

Is this a thing that people do? Does it have a name? Is it worth talking about?

Maybe something like this will be in the "communicating with stakeholders" section. I feel like knowing the rate at which how many tests are going from failed to passed is something that could be useful to stakeholders to gauge completeness... ?

Ensure all chapters receive thorough review.

I would hate to see a situation where the first several chapters are thoroughly reviewed while the later chapters get short shrift. It would probably be best to have at least a couple different pairs of eyes on each chapter, in addition to your own.

Feel free to use and edit this list to keep track of which chapters have been reviewed in detail, and by which reviewers, so that hopefully our collective resources can best be allocated. The checkbox doesn't so much mean "done" as simply "more done than the others".

Setting the reader up for Ruby/Haskell love that isn't there

File 5, line 31 says:

I'll often mention how much easier something is to do in Ruby, or how a particular kind of defect doesn't even exist in Haskell, or other references along those lines.

which is great, and all; except... in this book, you kinda don't. Go ahead and grep -irn --color '\(Ruby\|Haskell\)' text to see for yourself.

While you do mention Ruby and Haskell a few times each throughout the text, it tends to be for other reasons. It may make sense to retool this sentence to be a bit more accurately reflective of how Ruby and Haskell are actually portrayed in this book.

Lest you be tempted to just take that sentence out altogether, I will add that I do think it's important for something to be said along these lines.

Tic Tac Toe is underspecified

I'd have my pedantic nerd card revoked if I didn't point out that a tic tac toe board can have BOTH no open squares AND an entire row, column, or diagonal filled in with the same marker, thus leaving an ambiguous case in your specification that currently reads:

Players shall then take turns placing X’s and O’s (by the first and second player, respectively) in open squares, until there are no open squares (in which case the game is a draw) or an entire row, column, or diagonal are filled in with the same marker, in which case the owner of that marker (X for first player, O for second player) shall be the winner and the other player shall be the loser.

And I'm only 3 pedantry points away from a free coffee.

Ambiguous sentence - not sure how to reword

In section 7.1:

If nothing is preventing odd input coming in, at some point it probably will.

My sleepy brain resolved this sentence to mean "If nothing is preventing odd input coming in, at some point the system probably will prevent odd input coming in" rather than the intended "If nothing is preventing odd input coming in, at some point odd input probably will come in".

My sleepy brain is unable to come up with a sufficiently pithy yet unambiguous way to reword this, though... "If nothing is preventing odd input coming in, at some point War and Peace probably will come in as a product name", "If nothing is preventing odd input coming in, at some point it will probably be like living under a leaky roof"...?

Meh. You're the writer here, so I'm leaving this one to you rather than submitting a correction :)

compile failure

Linux ubuntu 3.16.0-38-generic #52~14.04.1-Ubuntu SMP Fri May 8 09:43:57 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
pandoc 1.12.2.1 Compiled with texmath 0.6.5.2, highlighting-kate 0.5.5.1.
pdfTeX 3.1415926-2.5-1.40.14 (TeX Live 2013/Debian) kpathsea version 6.1.1
XeTeX 3.1415926-2.5-0.9999.3-2014012222 (TeX Live 2013/Debian) kpathsea version 6.1.1

Dump:
...pandoc text/98_glossary_of_terms.md
--listings --highlight-style kate
-o compiled_tex/98_glossary_of_terms.tex
pandoc text/99_acknowledgements.md
--listings --highlight-style kate
-o compiled_tex/99_acknowledgements.tex
pandoc main.tex -o software-testing-laboon-ebook.pdf
--latex-engine xelatex
--chapters -N --toc --toc-depth=2
-M documentclass="book"
-M classoption="twoside"
-M classoption="letterpaper"
-M geometry="margin=1.25in"
-M title="\Huge{\textbf{A Friendly Introduction\to Software Testing}}"
-M author="\LARGE{Bill Laboon}"
-M date="{\Large\textcopyright~\textbf{2015-08-24}}\Compiled in PDF\LaTeX{}\endgraf\textit{for AKS and CKN}"
pandoc:
Error at "source" (line 423, column 68):
unexpected "\n"
expecting "!"
A string of non-alphabetic characters, e.g. \lstinline!78 &^\item
^
make: *** [pdf] Error 1

"DRY" is used colloquially before it is defined

File 45, line 259 talks about how we can further refactor the extended example by "DRY"-ing up code; however "DRY" isn't defined until the following chapter, where a whole section is devoted to the concept. The simplest solution may be to avoid invoking the term "DRY" here.

Additional argument against testing private methods: inhibits refactoring

My personal argument against testing private methods is missing from the flame war presented; I can only assume this is an oversight ;)

IMO, the reason you use private methods is so that you reserve the right to, in the future, aggressively refactor the internals of that class and change all the private methods around willy-nilly. If you add tests for your private methods, though, you've now created a barrier to that refactoring-- you're either going to have to update all those tests or you did work that you're going to delete when you refactor.

"logged on" sounds weird to me, possible personal problem

In section 6.1.6, the phrase "logged on" is used twice:

This may have been accomplished in a variety of ways: the user has never logged on before; the user is already logged on and bought any items that were in the cart, resetting the counter;

This may just be me, but it has an association in my brain with early 90s internet terminology, like "surf the world wide web" and sounds a bit dated/out of touch. Or a bit hollywood:

swordfish

"Logged in" would sound fine here, if you were talking about a system that a user would have to authenticate to, which may not be the case (and I'm presuming is the reason you used "logged on" instead). I'd probably say "the user has never visited the site before; the user has already visited the site" if we're talking about a web-based e-commerce system, or "the user has never used this software before; the user has already used this software" if the intent is to be more general.

Or tell me to sit on it :)

Cite a source for "overweight" BMI changes

In section 8.3, you mention:

In 1998, tens of millions of people suddenly became overweight due to a redefinition of the term “overweight” in regards to body-mass index (BMI).

I think this deserves a link or something.

Suggestion for test case status example

You might address this later in the book, or you might have decided not to address this, but what about a test case that depends upon a temporal state that we can't mock? For example, a test case that can only be run during full moons, daylight savings time switches, or in the last ten minutes of every hour?

Would this fall under the "blocked" status, if you were doing a test run at a time other than the one needed? Is this worth calling out?

Dependency injection clarification

In 14.1 Test Doubles, right before the Dog example, you say:

In order to minimize issues, you should pass in the object as a parameter whenever possible, as opposed to relying on class-level variables or even worse, global variables.

But a bit later, you say:

It would be even more difficult if _df did not have a nice setter function, but instead was created
internally or only as a byproduct of an entirely different method.

This case of creating an object internally is actually more common in my experience than relying on class-level variables or global variables and I think deserves being called out in the earlier sentence.

Unless I'm just misunderstanding the terminology? In Ruby, the case not mentioned would be described as "instance variables" or fields or I do like the description "internally", while class-level variables are something different....?

Lack of cohesion at 13:18

File 13, line 18 lacks cohesion:

Remember, though, that a CEO is in charge of an entire corporation, not just engineering, IT or any other particular part. There's an entire company to run, and even for a tech company, software testing will be a relatively minuscule part of the company in most cases. There are many other areas asking for resources, often with good reason. The marketing department is asking for more marketers, and a company that doesn't come up with good ideas for products will fail. Operations is asking for more help desk personnel; if people can't figure out to use the product, they will stop buying it and the company will fail. All that being said, there are numerous reasons for that CEO to turn down the project manager's plan.

Is this continuing the argument from the previous paragraph that the CEO should seriously consider cutting testing, or is this offering the more sane contrasting opinion? If both, consider expanding and splitting into two paragraphs.

Impact on the *user base*, not just one user?

In section 9.3.6 about defect impact, what do you think about discussing assessing/mentioning the impact to the user base as a whole, rather than just the impact to one user? That's definitely influenced my bug triaging-- are we talking 1 user, 10 users, IE users, all the users in the most important customer, users who log in more than once a week, etc?

2 different implementations of `haveFunAtDuckPond` is confusing

So in chapter 13 Intro to Unit Testing, haveFunAtDuckPond is defined as:

public void haveFunAtDuckPond(DuckPond duckPond) {
    System.out.println("Having fun at the duck pond");
    duckPond.haveFun();
}

in order to illustrate the difficulty of testing impure functions.

Then, at the beginning of chapter 14 Advanced Unit Testing, you have this implementation of haveFunAtDuckPond:

public int haveFunAtDuckPond(Duck d, int numFeedings) {
    if (d == null || numFeedings <= 0) { return 0; }
    int amountOfFun = 0;
    for (int j=0; j < numFeedings; j++) {
        amountOfFun++;
        d.feed();
        d.quack();
    }
    return amountOfFun;
}

This one has a return value in addition to some side effects.

THEN, later in chapter 14 in section 14.3 on Mocks and Verification, it says:

"Yes, yes, this is all fine," I can hear you saying, "but you didn't answer the original question! We are still dependent on asserting on a value that is returned from a method, and thus won't be able to test methods without a return value!" Remember the method that we wanted to test from earlier in the chapter:

public class Person {

    public void haveFunAtDuckPond(DuckPond duckPond) {
        duckPond.haveFun();
    }

}

There is no return value and thus nothing on which to assert.

So for one thing, this implementation is not from earlier in this chapter, it's from earlier in the book or from the previous chapter.

For another thing, the intervening reimplementation of haveFunAtDuckPond between this reference and its antecedent is confusing... which one am I supposed to be remembering? Why are there 2?

I'd consider making them be two separate concepts entirely and only have one implementation of haveFunAtDuckPond in the book, even though in the real world there are multiple ways one could have fun at a duck pond (not stepping in duck poop is my definition of having fun at a duck pond ;)

Second definition for "verification" needed

File 10, line 53, defines "verification":

This illustrates the difference between verification and validation. Verification is ensuring that you're building the software right; validation is ensuring that you're building the right software.

However, file 41, line 210, offers a second definition:

In the above instance, instead of asserting that a particular value is returned (since no value is ever returned), you instead can make a "meta-assertion" that .haveFun() is called. This is called verification since you are verifying that a method has been called. Note that this kind of verification has nothing to do with the kind of verification that means "checking that the software is right". They are two different concepts that happen to share the same word.

Since it is bolded as a vocabulary word, this second definition of verification, with respect to mocks, should appear in the glossary. It should probably be its own entry, with an appropriate parenthetical.

BADREQ vs. REQ

In section 5.1, BADREQ-1 is used to label a "bad requirement", but other similarly bad requirements in the surrounding text are labeled as REQ-1 and REQ-2.

Code coverage is even worse than you describe

You say:

Having a statement or method covered by tests does not mean that all defects in that unit of code have been found by the test!

But it's EVEN WORSE than that-- having a statement or method executed during tests does not even mean that you asserted on ANYTHING having to do with that code at all!!

Please instruct your students that they can easily win the adoration of managers demanding 100% method coverage AND passing tests simply by calling every method from tests but never asserting anything. I need more cohorts equipped with the knowledge to subvert bullshit developer metric targets.

Consider not promoting the term "sanity test"/"sanity check"

So, for one thing, I'm not entirely convinced that there's a significant difference between how you've defined "smoke test" and "sanity test" in section 10.1.

For another, the term implies that "sanity" is a good thing and "insanity" is a bad thing, and, albeit indirectly, subtly reinforces the stigma associated with mental health problems.

While leaving the term "sanity test" out of the book certainly isn't going to stop everyone from using it, it's something that I thought you might be interested in at least thinking about.

On that note...

Warn against being *too general* in defect summaries?

Section 9.3.1 about defect summaries is mostly about being too long and specific; but I think being too general could be a problem as well. If all the bugs in the list are, e.g., "search is broken", "search totally broken", "can't search", "search doesn't work", that doesn't really help much either.

Define "feasible".

48a9b83 bolded "qualitative" and "feasible" (per @carols10cents' suggestion in #29) for internal consistency with "quantitative".

Since "quantitative" and "qualitative" have glossary entries, "feasible" should, too.

Single fancy quote is going the wrong way, once

In the rendered PDF version built 2015-08-21, in section 6.1.5, one of the single smart/fancy/pretty whatever quotes, the first one around the first a, is going the wrong way as shown in the screenshot; the incorrect one is highlighted in red and other quotes that are displayed as I would expect are highlighted in green.

screen shot 2015-08-17 at 2 00 45 pm

All of these characters appear to be ASCII 39 in the markdown... sooooo.... something in the md -> pdf toolchain must be doing it...

Consider condensing or dropping section on testing private methods in java?

Advanced unit testing chapter 14 spends a lot of time building up to how to test private methods in java using reflection, and I'm not sure that it merits that much inclusion in the context of the whole book.

4.5 pages out of 211 is a full 2% of the book (and I didn't even exclude the cover pages or glossaries, so it's an even higher percentage of the actual text). The section meanders through explaining what reflection is, takes a detour through code that only works for public methods, mentions that we've basically implemented a REPL (and then has to explain what REPLs are and why they're useful, which is neat but not particularly relevant), and then gets back to actually testing private methods. And all this for something that in the previous section, you said you mostly didn't recommend unless the existing code meant you had no other choice!

This is also pretty language-specific in a book that claimed in the beginning to be using Java just as something that's pretty universal that everyone could figure out. It's in stark contrast to the other concepts in this chapter like doubles, stubs, mocks, setup, etc that are generalizable.

However, if this is a concept/question that comes up in class that you spend time on and people find worthwhile, I could see justification for leaving it in. This is the Advanced Unit Testing chapter, after all...

Maybe condense this section to just the last example testPrivateLaboonify and a light explanation? Or relegate this whole section to an appendix? I wouldn't shed tears if it was gone completely though ;)

Contradiction at 25:53

File 25, line 53 has a contradiction:

In this field, a summary of what the test case is supposed to test, and how, is provided. In this way, someone reviewing the test case can tell at a glance what is meant to be tested, and sometimes how. It is usually possible to determine this by a careful examination of the preconditions, input values, and execution steps, but it is usually easier for a human to just read what the test is supposed to do.

"How" to test something is to be provided in the test case summary field, but someone reviewing it can only "sometimes" tell how to test it?

Drill Sergeant metaphor a bit harsh

In Section 2.1: "but it’s not going to get any stronger until someone tries to hit it with some discipline."

In this metaphor, the function is a child (based on the usage of "helicopter parent"), so it's kind of alluding to hitting a child.

Bug or defect?

I'm guessing that you changed between preferring "bug" or "defect"? Section 2.4 has "bug" in its title, but that section only ever uses the word "defect" and never says "bug". Given that this is very early on in the book and you haven't defined everything yet, I think it would be better to do one of the following:

  • pick one of bug or defect and only use that one
  • explicitly state that there is a difference if you think there is
  • explicitly state that you're going to use "bug" and "defect" interchangeably if you think there isn't a difference

Pages intentionally left blank?

There are multiple pages at the end of chapters that don't have any content on the page other than the page number and the chapter heading. Some chapters end as I would expect in a partial page.

Example of a blank page:

In the versions of the PDF generated on 2015-08-27 and 2015-08-21, the last page of Chapter 14 Advanced Unit Testing is totally blank (it's page 126 on the 27th and page 124 on the 21st). The fact that it's happening in 2 different versions makes me suspect that there's something in the PDF generation toolchain responsible.

Example of a partial page:

In the versions of the PDF generated on 2015-08-27 and 2015-08-21, the last page of Chapter 2 What Is Software Testing ends in a partial page (16 in both versions).

I wonder if it's trying to do something like not start a chapter on the right hand side if this were printed double-sided? But all the page numbers are even...

If this is intentional or acceptable, carry on :)

Confusing statement concerning stubbing

Section 14.2: "Note that we can even stub methods that don’t exist, thus allowing us to not only test
classes that have errors."

This doesn't make much sense in the Java world of things, since you can only stub methods that are at least declared on the class/interface you're mocking. Maybe you can incorporate a Ruby example

Distinguish between before/after method ordering and test ordering

In this paragraph in 14.4 setup and teardown, I feel like 2 different concepts are being muddled:

Also note that while there is nothing stopping you from prefacing multiple methods with @Before or @After annotations, it's usually not necessary and will just make it more difficult to follow the code. If you do make multiple methods with these annotations, they will run in a deterministic order (that is, if you run it again, they will run in the same order). However, since tests should not depend upon each other, this ordering is not specified and is liable to change whenever you update your code and/or JUnit version. It is possible to specify the order that tests will run in, using the @FixMethodOrder annotation, but you want to avoid doing that if at all possible. Honestly, I feel a little guilty even writing down that it is possible.

You're discussing first the order in which multiple before/after methods run and then you switch to talking about the order that tests will run in. Maybe these are the same thing in jUnit? But I don't think they are, since the "deterministic order" describes the before/after methods as running in the same order if you run the tests again, whereas I hope jUnit is randomizing the order in which the tests run differently for every test run, otherwise it would be less likely that you'd see failures due to a dependence on a test ordering?

So if you use the @FixMethodOrder annotation, will that affect the order of the before/after methods too or not? I can't tell from what you've said here.

I think the test ordering could be left out here and perhaps discussed elsewhere instead... or perhaps just leave this whole paragraph at "Also note that while there is nothing stopping you from prefacing multiple methods with @Before or @After annotations, it's usually not necessary and will just make it more difficult to follow the code."?

Is test plan template immediately followed by definitions necessary?

It seems a little redundant to have the test plan template in section 6.1 with just the field names and empty space to the right of them, and then immediately repeat the field names with their definitions in section 6.1.1. Especially given that the field names are bolded in 6.1.1, I think it would be fine to just have that and leave out the blank template in 6.1... but perhaps you have a reason for this :)

Link to Franklin's nil talk

In section 7.1, under Null Pointer Errors. Bonus points if you can create a link to someplace that will be available in perpetuity.

More unfinished lines.

BEEP BEEP. Thanks to @WillEngler for the idea in #19.

I used some serious pcregrep-age to find these and sifted through a bunch of false positives, so hopefully (fingers crossed) this is it.


File 32, line 39:

With online systems, oftentimes beta testing can be "mixed in" with regular usage of the system. You can have a specific subset of users - say, those who have shown an interest in trying new features, or customers important for revenue, or even a random sampling - have access to the new functionality you wish to beta test. Usually, you want to either clear it with these customers or make sure that they know that the functionality they are using is not considered complete. You can then BEEP BEEP.

File 50, line 118:

While the previous section described how to ensure code that was testable from a white-box perspective, creating a program that is easily testable from a black-box, systems-level perspective can be even more difficult. Methods, in general, are meant to be called - there's a built-in interface to do it, and the parameters for each method are generally specified (there are ex BEEP BEEP.

File 70, line 37:

By selecting the most important performance indicators - referred to as key performance indicators or KPIs - you can focus on testing the aspects of performance most important to the users. This will then allow developers to tune the system to meet the needs of the users, instead of wasting time on parts of the system which the users will not tend to care about, or already have a sufficient level of performance that spending more time on improving it would be a waste of time. Depending on the level of specificity of the requirements, it may be possible to determine BEEP BEEP.

File 98, line 59:

Combinatorial testing: Testing in such a way so as to ensure that various combinations of variables will work as expected. For example, testing a BEEP BEEP.

Math is hard

I don't think this math is right, but I'm not sure... this is in 12.1, Benefits and Drawbacks of Manual and Automated Testing, under Benefits of Automated Testing, #8 It scales very well:

Adding automated tests often results in a sub-linear increase in time, since much of the time for automated tests often consists of setup time. That is, adding five test cases to an existing plan consisting of ten cases may result in an execution time that is much less than twice the original test plan.

Ok. So if we start from 10 cases, and let's say each test takes a second to run, so that's 10 seconds. Scaling linearly would mean that if we add 5 cases, the test run would take 15 seconds. 15 seconds is much less than 20 (twice the original 10 seconds), but I think you're trying to say that it scales less than linearly, so it would be much less than 15 seconds, which is not twice the original 10 seconds.

If i'm understanding everything correctly, and I'm not sure about that so that's why this is an issue and not a PR, I think you could change this to:

That is, adding ten test cases to an existing plan consisting of ten cases may result in an execution time that is much less than twice the original test plan.

or:

That is, adding five test cases to an existing plan consisting of ten cases may result in an execution time that is much less than 150% of the original test plan.

Maybe???

Advanced Unit Testing reorder

I think "Unit Test Structure" (and maybe also "Code Coverage" and "Unit Testing as Part...") should precede the sections on doubles/stubs/verification/etc., as you kind of implicitly use the knowledge in the "Unit Test Structure" when structuring the unit test examples in the preceding chapters. The flow would be more "this is how unit tests are generally structured" into specifics, rather than throwing out a bunch of low-level specifics then presenting the more basic concepts of unit test structure afterwards

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.