If any one else was confused by the poorly written article, he does the following.
* Download list of password hashes.
* Create list of seed words based on the subject matter for the site the passwords originate from (like "Afghanistan").
* Use Twitter search to build a dictionary of words related to the seed list.
* Feed dictionary through a script, john, that performs common letter substitutions and checks the md5 for each candidate against the original list of hashes.
I'm skeptical as to whether this is more effective than using the 4,400 most common English words.
Would it necessarily have more matches than a generic password dictionary? That would assume that military users are smart enough to not use the type of password they'd use on their personal email, but still stupid enough to use a dictionary word as a base.
He also didn't filter out non-military words. His 4,400 words would overlap considerably with list of common English words and could have been built by starting from any seed list.
I think it is an interesting concept. I can imagine this uncovering some jargon that is uncommon outside the niche, but unfortunately his experiments don't prove anything.
I would be interested in hearing the results based on derivations of just a single word, 'password' - I bet he could have nailed more than 2000 passwords on the common mashing of that word (123password, passwordabc, etc...)
Repeating his procedure using a word list consisting of just the word "password" (according to him the --rules option trys a bunch of variations) yields 18 passwords.
I thought the article was excellently written, one of the best, with clear screenshots and explanations of what he's doing. (I do code though.)
others here have easily replicated his procedure to compare with alternative ones (e.g. just based on the word 'password'), and attacked his methodology (he should have compared with twitter searches unrelated to the military to show what he's claiming, the effectiveness of targeted twitter searches).
this is great. it means his methodology is easy to understand/follow, to verify, attack/question, modify, and reproduce. it's extremely clear what he's doing, and what he's saying. what more could you ask?
So I was curious and replicated the experiment. Here is a summary of the results.
* Twitter word list: ~2,000 passwords guessed
* Just the word "password": 18 passwords guessed
* Ubuntu words file: ~10,000 passwords guessed
I couldn't find an easily-downloaded list of the most common English words, maybe that would work even better.
Note that the words file has over 200,000 words in it, whereas the twitter list had only ~4,500. So there could be merit to this method. Of course a ton of those words are trivial ("A" and "a" are both there, for example), so who knows.
Update: I found a list of common words that was easy to get into the proper format. It purports to be the 978 most common words, though it seems to have been compiled in the '60s. Whatever, close enough for me. It resulted in 893 hits. So, here's the breakdown:
* Twitter (4,674 words): 2,198 successes
* Just "password" (1 word): 18 successes
* Ubuntu words file (234,937 words): 11,969 successes
* Common words (978 words): 893 successes
* Default wordlist (?): 102,297 and still going...
This is based on 163,636 from the same database the author used.
Or, you know, if you try to brute force 180k accounts, you're bound to get 2000 of them by chance alone. I'd like to see him change the words to food-related items and see how many he gets.
This is a great point. If the hypothesis is "unique words gathered from Twitter search based on queries to find terms related to the military", then that's easily falsifiable, like you said.
I wish that more people running these kinds of "experiments" went to the extra step of trying to falsify their hypothesis like proper experimentation requires.
I see no explanation for why this should be better than just using an English dictionary or some other corpus. A comparison with a couple of easily found corpuses like the standard Linux words file would have been interesting.
(Although, obviously, using the Twitter search method is more targeted and will likely give you results this century when you're looking for something specific.)
Google is only returning three for me right now. One of the profiles spells adult "aduld." But more importantly 3 out of how many profiles? Thanks for the empirical data showing that the mispelling is a poor choice for a base word list.
* Download list of password hashes.
* Create list of seed words based on the subject matter for the site the passwords originate from (like "Afghanistan").
* Use Twitter search to build a dictionary of words related to the seed list.
* Feed dictionary through a script, john, that performs common letter substitutions and checks the md5 for each candidate against the original list of hashes.
I'm skeptical as to whether this is more effective than using the 4,400 most common English words.