Critical Thinking 101: WSJ #2

So, how is it that the Wall Street Journal’s Full Methodology flunks the basic, pass-fail test? Why do all the numbers in “On Web, Children Face Intensive Tracking” add up to a zero-value story?

The Full Methodology is disappointing on several counts, but the core problem is related to Principle #3 in the previous post: Whenever we read numbers, where something has been counted, it’s important to ask, How were things that could affect the count controlled?

The Full Methodology tells us this:

Mr. Campbell used Mozilla Firefox 3.5 and Adobe Flash Player 10.0. Following each session, he examined the tracking files that had been placed on the computer.

That’s all we’re told about the browser. We don’t know the main thing: How was the browser’s cookie management configured?

Which browser was used is of relatively minor importance. I happen to have IE6 on the computer I’m using now, and even that relic allows blocking of either third-party cookies or first-party cookies, or both. Firefox 3.6, which I actually use, has much more granular cookie handling functionality. I can allow cookies only from certain sites, or block them only from certain sites. Or block all 3rd party cookies, etc.

Since the research involved collecting cookies from websites and counting them, the main thing we want to know is how Mr. Campbell, the researcher, configured the browser for cookies. What gets caught in a net very much depends on what the net is designed to catch — minnows or tuna or whatever — and what the net is designed to catch is analogous to how the user has decided to configure his or her browser. If you’re browsing a website, what you accept in the way of cookies in under your control, not the website’s.[1]

There is a clue to how Mr. Campbell, configured his browser in the legend for a graph on the WSJ site, “What we found on one site”. The site is Snazzyspace.com. There were 195 cookies and 4 first party cookies…

Pause there.

If they know there were 4 first-party cookies, and there were 195 total cookies, there must have been 191 third-party cookies, and to know that, the browser must have been set to accept third-party cookies. Mr. Campbell could have blocked 191 cookies simply by de-selecting the “Accept third-party cookies” box in Firefox.[2]

But four measly cookies wouldn’t make a sensational story would it? Four cookies from hitting 20 pages on a website just doesn’t rate front-page play. Is it possible that… Did the Journal promote 10 inches worth of filler from inside Weekend Journal to a front-page splash by telling Mr. Campbell to set his browser to accept 3rd party cookies?

PB

————–

[1] A website might not work if you block all cookies, which doesn’t change the fact that you’re in control. A discussion of cookie management can get quite involved. What I’m considering here are not flash cookies, but the old-fashioned, 90s technology flat text cookies we’re all familiar with, which in recent versions of Firefox are kept in a sqlite database.

[2] I thought the default in Firefox was to block third-party cookies, which means to get all those 191 cookies you’d need to deliberately change the setting.  I need to check this.

Posted in Uncategorized | Leave a comment

haegeum

just rehearsing

a tombstone.

Posted in Uncategorized | Leave a comment

Critical Thinking 101: WSJ #1

Today’s Wall Street Journal story “On Web, Children Face Intensive Tracking“, is interesting, and if you go to the website and click around you find it has a fair bit of material that fills in gaps. There’s a decent video explaining cookies, for those who don’t already know how they work.

I went to the web version of this story  because the paper version said this:

(Full methodology, as well as previous privacy investigations in this series, at wsj.com/WTK.)

I wanted to know about the “full methodology” for the study, because the article has several precise figures for the number of “tracking tools” placed on the Journal’s test computer. Precise figures suggest a methodology rigorous enough to produce reliable, precise figures:

…Y8 installed 69 tracking files…

…On average, the eight installed 81 tracking tools, close to the 82 average for all 50 sites…

…a games site called Shockwave.com, installed 146; another game-and-video site, nick.com, installed 92…

…the math-games site coolmath4kids.com installed 60 on a test computer…

…Weeworld.com installed 144 tracking tools in the Journal’s test…

Here’s the Critical Thinking 101 question for the day: What do you need to know about the Full Methodology of the research before you can gauge the importance of all those exact numbers?

Here’s what I think are basic, pass-fail, questions the Full Methodology information must answer:

1. What did they count?

In this case, it’s not difficult. We know what cookies are, since they’ve been around for a long time, since Windows 95 days. Flash cookies and beacons, we’ll guess, are different from traditional flat text cookies, but still discrete, countable things. In other words, if you looked at a bunch of them together there wouldn’t be any doubt as to whether there were 10 or only 8.

Principle #1: Whenever we read numbers, where things have been counted, it’s important to ask, What exactly did they count?

2. How many pages were pulled back from each site and how were those pages selected? Did the researchers just click on links randomly or did they have an algorithm for selecting links?

It’s an obvious possibility that the more pages someone clicks through, the more cookies will be collected, and different kinds of pages could serve cookies differently.

Principle #2: Whenever we read numbers, where things have been counted, it’s important to ask, How did they count?

3. What were the browser (and flash) settings?

Cookie intake is largely controllable by the user.[1] With the browser I’m using presently (Firefox 3.6), I can allow or not allow 3rd party cookies; accept cookies in general, but make exceptions for particular sites I want to deny; deny cookies in general but allow them for particular sites; flush them when I close the browser… It’s fairly granular control, and that’s just with basic browser configuration settings; other functionality is available with plugins or 3rd party software, and if I’m especially fastidious there are tricksy things I can do with my home network firewall, or with the hosts file on my computer.

Given that cookies slurped off the web are controlled by the user, how did the Wall Street Journal’s user doing research set the controls?

Principle #3: Whenever we read numbers, where things have been counted, it’s important to ask, How were things that could affect the count controlled?

With those 3 questions and principles in mind, how does the WSJ’s Full Methodology score?

Not very well, I’m afraid. In fact, it flunks.

But, the fact that the WSJ offered something intended to be a “full methodology” for the research at all, and referenced it in the paper edition, is very much to the Journal’s credit. In general interest media, studies are often cited, but information about how these studies are done — study methodology — is so rare that anything at all about the methodology of a study is worth noting, and even celebrating. With that in view, we can give the WSJ a few points for at least showing up[2], which pulls its grade up from a solid F to, say, a weak D+.

PB

—————–
[1] Yes, I know that Adobe’s ubiquitous flash software has made absolute control of all cookies somewhat more complex than it was 10 years ago.

[2] In many situations in life, you may not excel, but you can pass if you just show up.

Posted in Uncategorized | Leave a comment

Defending a new domain 5

“Like an answer, the three slogans on the white face of the Ministry of Truth came back at him…” -George Orwell

William J. Lynn III made this statement:

In cyberspace, the offense has the upper hand. [1]

As noted in a previous post, Mr. Lynn doesn’t bother to make a serious argument in support of this dubious theory. He does preface the assertion with a paragraph that begins “First, cyberwarfare is asymmetric”. But asymmetric warfare is what you have when an unambiguously superior force is dealing with pin prick annoyances at a tactical level. It has nothing to do with whether there is an intrinsic strategic advantage for the offense in a contest between equally matched opponents.

Mr. Lynn doesn’t make any effort to convince readers that “in cyberspace, the offense has the upper hand,” but as his essay develops he does refer to the theory, as if it were a settled fact:

In an offense-dominated environment, a fortress mentality will not work.

Given the dominance of offense in cyberspace…

Now, I’ve got a question for you: How would you complete that second sentence?

Given the dominance of offense in cyberspace _________________.

Well, logically, you’d insert something like, “it is imperative that the United States have a strong offensive posture” or “we must invest resources in offensive capabilities” or “we must accept the possibility that to win a conflict in cyberspace we will need to strike first”.

Given the dominance of offense, the side you want to be on is the side that goes on the offense, right? Let’s say, it’s football and it’s the Pittsburgh Steelers vs. Herndon High School. Complete this sentence: Given the dominance of the Steelers in football, I should place a large bet on __________.

If you say, “Herndon HS”, read no further. I’ve lost you.

You will not believe how Mr. Lynn completes the sentence:

Given the dominance of offense in cyberspace, U.S. defenses need to be dynamic. [emphasis added]

I understand that to mean, essentially, “Because A can beat the fool out of B, the U.S. should go with B shaking its hips and making funny faces.”

If you read this essay closely, you start to wonder if Mr. Lynn really wrote it. That is, you wonder if any one person wrote it. There are odd inconsistencies and logical gaps, context switches, subtle vagaries…

Certainly a government bureaucrat could produce 10 or 12 pages of text with logical issues. I understand that. But this essay reads more like the product of a committee, cobbled together over a long period of time, with compromises in selection of words and phrases — compromises that have nothing at all to do with network security (or “cybersecurity”, if you’re a jargon addict) and everything to do with some bizarre high-wire balancing act performed in the Circus where those ridiculous clowns we elect to public office gather to devise new con games for taking away what we work hard to earn…

But… I digress  : – )

At any rate, the numerous incoherent threads running through this essay can be nicely summarized by this one incoherent statement: “Given the dominance of offense in cyberspace, U.S. defenses need to be dynamic.”

An inquisitive reader, W. Smith, asks:

“So, where you expect the word “offense” Mr. Lynn uses a curious phrase, “dynamic defense”?

Yes, that’s right.

Smith: “And as the essay continues, does Mr. Lynn come back to the concept of dynamic defense?”

Yes, he does.

Smith: “Does he define it?”

No, he doesn’t.

Smith: “I see.”

Ah… I’m not sure I see, but… this is reminding me of something… surely it’s not… The B Vocabulary?

Hold on… Where did I leave my copy of 1984?

PB

——-
[1] Foreign Affairs, September/October 2010: “Defending a New Domain”

Posted in Uncategorized | Leave a comment

Defending a new domain 4

William J. Lynn III makes this statement:

In cyberspace, the offense has the upper hand. [1]

And the reason for this?

The Internet was designed to be collaborative and rapidly expandable and to have low barriers to technological innovation; security and identity management were lower priorities. For these structural reasons the U.S. government’s ability to defend its networks always lags behind its adversaries’ ability to exploit U.S. networks’ weaknesses.

His summary of the “structural reasons” is simply pitiful, but given what he says, what is to prevent “low barriers to technological innovation” from including technical innovations that improve security?

Nothing. In fact, that’s exactly what has happened. Where a need appears, people innovate.

As the internet has evolved from the days when a single hosts file was retrieved periodically by the few hundred computers linked in the network, the need for and importance of security has grown, so there has been tremendous, inexorable technical innovation in the area of security. Security oriented tools — firewalls, virus scanners, spam filters, wireless encryption algorithms, etc, etc… — have proliferated and become highly sophisticated. And the way we do things has changed. Do you still attach a Windows box directly to the net with file and print sharing enabled? I’ll bet you don’t. Your 12-year old getting familiar with linux knows ssh, but if she’s heard of it at all, she thinks “telnet” has something to do with fat televisions, like the one in the attic.

If Mr. Lynn wanted to identify a “structural reason” that could plausibly explain why offense would have the upper hand, he could have used this paragraph from later in his essay:

On average, it takes the Pentagon 81 months to make a new computer system operational after it is first funded. Taking into account the growth of computing power suggested by Moore’s law, this means that by the time systems are delivered, they are at least four generations behind the state of the art.

Yes, that could explain why the U.S. military, on defense, would be at a disadvantage with respect to, say, 4 or 5 Russian teen-agers taking the offense.

But using that as an explanation for a special advantage enjoyed by the offense in cyberspace, is problematic: it doesn’t explain why offense per se is superior to defense per se. It would support the idea that a technically lagging network will be at a disadvantage, defensively, against a relatively cutting edge offense. And it would explain the “always” in this statement: “...the U.S. government’s ability to defend its networks always lags behind its adversaries’ ability to exploit U.S. networks’ weaknesses.

A poorly defended network will always be at a disadvantage when up against clever and energetic attackers, and perhaps even against an attacker with mediocre skills who happens to get lucky. The “significant compromise” Mr. Lynn describes in his essay’s introduction could be cited as an example of an indifferent offense that overwhelmed the military’s inept network defenses.

Does offense have a special advantage over defense in “cyberspace”?

No. The question doesn’t even make sense without particulars. The only reasonable response to that question is a clarifying question: What defense, and what offense?

Given some specific information about the defense and about the offense, you could argue for the advantage of one over the other, in a particular case.

Offense: worm exploiting Windows autorun
Defense: homogenous network of Microsoft operating systems, with autorun typically enabled
Prediction: Offense overwhelms defense initially; defense eventually wins because vulnerability is easily eliminated and unwanted files easily found.

That “Prediction” is what actually happened in DOD vs. Agent.btz.

PB

——

[1] Foreign Affairs, September/October 2010: “Defending a New Domain”

Posted in Uncategorized | Leave a comment

Defending a new domain 3

William J. Lynn III was asked a question by Wired writers about an alleged foreign intelligence connection to a run-of-the-mill Windows exploit that got loose on Department of Defense networks in 2008:

But what spy service would launch such a lame attack?

His response did not address the question:

“It isn’t the most capable threat, I agree with that,” Lynn replies. “But that kind of makes the point. If you had something of the kind of capability you described and we suffered a compromise as the result of it, it clearly means that we need to have a new strategic approach…”

Mr. Lynn’s answer is not about spy services, spy service motivations, spy service capabilities or anything else related to spy services. It’s about DOD strategic approach going forward. DOD strategic approach going forward might be fascinating, but to me, the original question is quite as interesting and merits a better answer: “What spy service would launch such a lame attack?”

Here are two theoretical approaches to an answer:

#1. A lame spy service, or lame non-state entity, would launch a lame attack. In this category would be Somalia, North Korea, feudal realms that show on the map as being part of Pakistan, Iraqi factions, etc. The idea here is that if it was an unsophisticated attack, it must be an unsophisticated attacker. The attacker was doing his best, but his best was lame.

But what if the attacker wasn’t doing his best? What if the “lameness” was calculated? If the attack were too sophisticated, DOD might not have noticed it, but if the attacker wanted DOD to notice that wouldn’t be so good.

#2. So, another possibility: It wasn’t a real “attack” at all; it was a probe, a sophisticated adversary’s investigation into readiness of the U.S. military, a way of getting some information about how the military might react to a real attack.

In the context of traditional combat a commander may send a small body of troops forward to probe the enemy. The objective isn’t to break through enemy lines or win a battle, but rather to get enough of a skirmish going to see what the strength of an opponent is, to learn something about the disposition of opposing forces and their agility in response.

Could Agent.btz have been a probe?

Personally, I doubt the worm had anything at all to do with an intelligence service, but setting aside that skepticism, if its source was a foreign spy service, and it was a probe, what did adversaries learn about DOD readiness?

1. DOD was unprepared for an exploit which was already familiar.

Windows autorun has been a known vector for worms since the 1990s. This suggests that DOD’s approach to network security is reactive rather than proactive. DOD is not looking around, is not surveying the landscape as it were; DOD isn’t analyzing known exploits with its own deployed software (and hardware) in mind.

2. DOD’s reaction was disproportionate and off target:

The Defense Department’s geeks are spooked by a rapidly spreading worm crawling across their networks. So they’ve suspended the use of so-called thumb drives, CDs, flash media cards, and all other removable data storage devices from their nets, to try to keep the worm from multiplying any further. [Wired Nov. 2008]

The Pentagon actually launched an “operation” to deal with the worm: Operation Buckshot Yankee.

But remember, this was just a Windows autorun worm. Why not turn off autorun on Windows boxes, then require that removable storage devices be checked before use? How to check a device? Turn off autorun on a Windows machine and that machine can now safely check for Agent.btz on a thumb drive. If you see it, delete AUTORUN.INF and executables that go with it (better: copy the files somewhere first; the more specimens the better).

3. DOD strategists went off on a snipe hunt.

This incident was “an important wake-up call,” according to Mr. Lynn. Two years later, his essay in Foreign Affairs suggests that after DOD woke up it invited a bunch of Hollywood script writers to start attending meetings with defense contractors and network security illiterate beaurocrats — to devise an expansive strategy that would include everyone and do everything.

First accomplishment of this team: slap a new label — “Cyber” — on everything that for more than a decade we’ve called “network security”. And jack up the price. Then, instead of doing things to improve existing network security operations, add on top of real network security work a thick layer of talkers, report writers, PowerPoint presenters and strategic thinkers. CyberExperts. Network security expertise: Not Required.
—–
How would an adversary use knowledge gained by an Agent.btz probe? One thought:

If the Pentagon, and the U.S. Government in general, is prone to over-react to network intrusion events, an intrusion doesn’t need to be effective in itself; it might be as lame as Agent.btz, with most of its impact dependent on U.S. over-reaction.

Here’s what I see: People who should approach real network security problems with balanced, thoughtful analysis, instead defer to sci-fi obsessed story tellers — who can be depended on to propose super-cool, super-scarey cyber-problems, even cyber-catastrophes, like financial system collapse and power grid failures.

Is all the wacky talk of cyber-this and cyber-that preparing our military, and more broadly, the U.S. Government, to react with hysterical frenzy to network security incidents?

Take a picture of this:

A geeky attacker wearing a funny hat but without a weapon jumps out from around a corner and shouts, “Boo!” The American soldier is startled, and seconds later he blows himself up fumbling with radio, gas mask, rifle, bayonette and grenade pins all at once…

In the wake of Agent.btz and the introduction of DOD’s extravagantly off-target cyberdefense strategy, an adversary may be thinking…

“They over-react… hmm… very interesting.”

PB

Posted in Uncategorized | Leave a comment

Defending a new domain 2

William J. Lynn III, in his Foreign Affairs essay (Defending a New Domain), says that this “previously classified incident was the most significant breach of U.S. military computers ever.”

Given the rather primitive exploit Mr. Lynn is writing about, a better way of saying that would be “…the most significant breach of U.S. military computers yet detected.”

In 2008, DOD was hammered by a variant of a Windows-only worm, Agent.btz, which uses Microsoft’s autorun functionality to spread. It’s a variant of Silly.FDC, which at the time was familiar, globally, to those interested in, or responsible for, network security. The concept of exploiting autorun goes back into the 90s. There was nothing new about the exploit and as far as I can tell from reports, the worm was not remarkably clever.

Agent.btz would be just another pedestrian Windows exploit if it hadn’t landed, probably by chance, in a very large and important network, perfectly suited to its simplistic technique for spreading itself.

Because Agent.btz spreads using Windows autorun, and because the military was disoriented and reeling in late 2008, DOD banned the use of thumb drives (a ban since moderated). The worm first got onto a DOD computer from a thumb drive, according to Mr. Lynn. But in a Windows-heavy network where sysadmins give autorun a blank check, mapped drives are far and away the primary vector for this kind of worm. One thumb drive infects one computer; one network share mapped by 1000 computers infects 1000 computers.

Although it didn’t spread on the network other than through drive mappings, Agent.btz did, apparently, attempt to download binaries from certain domains on the Internet, including a .cx domain (Christmas Island), and it’s possible those binaries had network-aware capabilities. About that, I don’t know. If the DOD security guys didn’t isolate and study a few instances of the worm, they also don’t know.

Mr. Lynn refers to this as a “significant compromise”. Let’s carefully parse out exactly what was significant:

1. The worm itself was NOT significant, but
2. DOD’s vulnerability to a garden variety Windows exploit was significant, and
3. DOD’s hysterical response was significant.

Mr. Lynn credits this “significant compromise” to the work of a foreign intelligence service, which on its face is unlikely. But in fairness to Mr. Lynn, we can assume that he himself honestly believes the statement he made. Perhaps he was shown evidence he found convincing.

If the chaps who gathered that evidence were the same Apple Dumpling Gang responsible for security on DOD networks at the time, the particulars of their evidence would warrant close inspection. The assertion that a foreign intelligence service was responsible is an interesting theory, but in the absence of evidence for or against that theory, I’m skeptical, albeit without a settled opinion. Mr. Lynn does believe the theory.

He was asked a pointed question about this foreign intelligence service connection by writers at Wired:

But what spy service would launch such a lame attack?

“It isn’t the most capable threat, I agree with that,” Lynn replies. “But that kind of makes the point. If you had something of the kind of capability you described and we suffered a compromise as the result of it, it clearly means that we need to have a new strategic approach and that’s what started a couple years ago. I’ve tried to lay out where we’re going going forward.” [Danger Room]

If you boil down the first paragraphs of his essay to the essence, Mr. Lynn is telling us that DOD got womped by a lame attack. It follows that DOD readiness and/or competence in the area of network security is the one issue needing attention.

Is DOD readiness and/or competence the focus of Mr. Lynn’s essay?

Unfortunately, not. Instead he wanders far and wide, across a landscape that becomes increasingly surreal: “the scale of cyberwarfare’s threat to U.S. National Security,” etc, “cyberwarfare is asymmetric,” so forth and so on, Cold War deterrence models don’t apply in cyberspace, “sophisticated intrusions into the networks that control critical civilian infrastructure…”, “…computer induced failures of U.S. power grids…”

Yikes!

Did he forget to take his meds?

I don’t think it’s that. It’s a style of thinking, and it’s a style of thinking not uncommon in government. To understand it, you have realize it has nothing to do with solving real-world problems. Any given “problem” is more analogous to a card drawn from the deck in a game of charades: You want your team to guess the word or phrase. What the phrase *is* doesn’t matter. Put a lamp shade over your head, hold your nose and hop on one foot, jump on a table and strike a muscle man pose… whatever you need to do. The idea is to inspire the right guess.

In government the idea is to get the right soundbites, get attention from the White House or from a key Senator, get funding for a project, get invitations to speak in the coolest venues, etc. The thing you start with is whatever was written on the card you drew. The thing on the card might, in fact, be a problem worth solving, but solving the problem and making headway in the game are two completely different and unrelated things.

Mr. Lynn’s Foreign Affairs essay is all about playing the game. If you read it expecting ideas for solving a problem worth solving — DOD’s backwardness in the area of network security — all you’ll see is a fantastic caricature of the Department of Defense.

But you need to know that the man spinning around on his back, kicking his feet in the air and flailing his arms, simply wants his team to shout out, “Beetle on its back!” He’s not trying to help all beetles or any particular beetle get to its feet. So, with that in mind, you can excuse Mr. Lynn for laying out “a new strategic approach” devoid of any strategy related to a real-world problem.

PB

Posted in Uncategorized | Leave a comment