atbash_cipher test suite is impossible(?) to satisfy about erlang HOT 10 CLOSED

Ah, I can't see nitpicks for problems I haven't submitted, but I'm still skeptical this is a solvable problem.

from erlang.

I'm pretty sure that decoding ignores word boundaries and just uses groups of five.

from erlang.

The test explicitly encodes and decodes the sentence, expecting the output to match the input. The specification is reasonable, but the test is not.

https://github.com/exercism/xerlang/blob/master/atbash-cipher/atbash_cipher_tests.erl#L16-L18 https://github.com/exercism/xerlang/blob/master/atbash-cipher/atbash_cipher_tests.erl#L16-L18

-John

On Aug 26, 2015, at 9:27 PM, Katrina Owen [email protected] wrote:

I'm pretty sure that decoding ignores word boundaries and just uses groups of five.

—
Reply to this email directly or view it on GitHub #62 (comment).

from erlang.

Oh, yepp. I see what you mean. This was not included in the original exercise. I would suggest that we remove it.

from erlang.

As I poke around the other tests, there's very little consistency on how to treat underspecified cases.

E.g., the Ruby test removed whitespace between integers, while the Erlang test doesn't even expect punctuation to be removed. The Ruby test expects capitals to be lower-cased when encoded, while the Erlang test expects them to remain upper-case.

I'm preparing a PR for this test to make it a little less slanted towards cases the developer can't possibly expect to solve without looking at the test in advance, but I'd suggest this problem in general get a much clearer specification or be dropped entirely.

from erlang.

Aha, I figured it out.

The tests ignore this part of the specification:

Ciphertext is written out in groups of fixed length, the traditional group size
being 5 letters, and punctuation is excluded.

(Update: specifically, I mean the Erlang tests.)

from erlang.

Anyway, I don't know the protocol here. Would completely dropping these tests (and tweaking the README, since it makes impossible statements like "Decoding gsvjf rxpyi ldmul cqfnk hlevi gsvoz abwlt gives The quick brown fox jumps over the lazy dog.") in favor of tests ported from another language be a problem?

It's unclear to me how Exercism manages "epochs" in problems; if one user solves them using one test suite, and another using a different test suite, I can see potential confusion when nitpicking. On the other hand, since the tests are checked out just once, a user shouldn't care all that much if things change later.

And I don't see people downloading others' code to run against the test, so I guess my concern is largely unjustified.

Thoughts?

from erlang.

Would completely dropping these tests (and tweaking the README [...]) in favor of tests ported from another language be a problem?

No, not at all. We're having a related discussion about clarity, purpose, focus, and consistency here: https://github.com/exercism/xjavascript/issues/105

It's unclear to me how Exercism manages "epochs" in problems

It's not been handled consistently. In most cases we've just sort of shrugged and decided that the point is to have an interesting and useful discussion, and that the code that is there is the code that is there. Unfortunately, we haven't communicated that clearly, which sometimes leads to people leading with "this doesn't pass the tests!", which is counter to the purpose of the site.

In a few tracks we've started introducing a simple versioning/book-keeping system, where we have a final test that asserts on some constant, and then the solution implements that constant. E.g.:

https://github.com/exercism/xruby/blob/master/hamming/hamming_test.rb#L81-L84

This at least gives nitpickers an indication that the version might be different from what they saw.

And I don't see people downloading others' code to run against the test, so I guess my concern is largely unjustified.

I do that sometimes. There's a hack for it in the CLI, too, because I was pretty fed up with copy/pasting:

exercism download SUBMISSION_ID

from erlang.

Since #66 is merged, can't this get closed?

from erlang.

agreed, thanks.

from erlang.