SoK: Science, Security, and the Elusive Goal of Security as a Scientific Pursuit

Measuring and analyzing security in a way that reliably shows progress is a unique and difficult challenge. Computer security research is unclear. This lack of clarity has led to a movement to put more science into security; or develop a “Science of Security.” However, exactly what scientific security research would, could or should look like is vague. Herley and van Oorschot offer some insight into this problem by highlighting aspects from the history and philosophy of science that are relevant to security research. They discuss the current science of security and how security research has not adopted lessons learnt from other sciences.

Inductive and deductive statements are different types of knowledge claims. Inductive statements are claims based on the observation of a real-world event and are inherently real. However, the claims are only true for those things observed and does not provide insight into things not yet observed. Alternatively, deductive claims are based on a self-evident truth and are absolute. The nature of a claim based on a preconceived premise is that it is abstract from reality. This means that it is not as applicable to the real world and is not able to explain anything new. Either kind of statement is compromised by being either true only for things observed or under certain conditions. There are no statements about the world that we can be absolutely certain are true. If no statements are certainly true then all statements could be considered as being equal, regardless of how sensible they are.

One way to overcome this problem, and make sensible statements, is to make the statement falsifiable. In other words, such a statement must be able to be proven to be wrong. This is what differentiates a scientific claim. Consequently, claims about things that cannot be observed or objectively tested are unscientific. Statements about things not physical such as religion, metaphysics or mathematics do not meet this criterion. Deductive methods that describe the world, such as geometry, may parallel the physical but are still separated concepts. The alignment between concept and reality must be tested to be confident about how it lines up in different situations. Scientific inductive statements are those that can be disproved. Scientific deductive statements are those that have an alignment to reality that can be disproved. Science is then able to be constructive, as something can be known and predictions about the unknown can be made. This understanding is consistent across much of the scientific world and is the basis of the method of: hypothesize, predict, validate.

Computers have evolved from a well understood and defined environment to a mess of interactions, unstructured data and adversarial input. It is now a chaotic environment not unlike the real world. Computer security has developed a great deal in the past four decades, but its scientific aspirations have suffered from disparate approaches. Security is often reliant on strict models or mathematical proofs that can fail when presented to the real world. When compared with other scientific fields, security research generally falls short by confusing inductive and deductive claims; making unfalsifiable claims, not testing; not making claims and assumptions clear; and seeking to confirm rather than to disprove claims. The security research community would benefit from a focus on core scientific elements, a greater commitment to clearly distinguishing between inductive and deductive statements to reduce confusion and rigorously testing the coupling between mathematical proofs of security and real-world systems.

Researchers should use evidence of effectiveness to justify security measures and avoid unfalsifiable claims or circular arguments. Properties of computing and its security may be unique but not so much as to exempt it from scientific approaches. A scientific approach does not mean that security should focus on laws or proofs like physics or cryptography. A prejudicial approach to developing security science could result in using inappropriate methods resulting in precisely measured, mathematically proven falsehoods. A harmonized application of both theory and measurement to security are needed for progress on the diverse set of problems in security research. This science should be attentive to unsupported assertions, undocumented assumptions and authority-based arguments while prioritizing efforts at refutation with evidence-supported statements. Further, placing research in context reduces the risk of providing orphan security components that never fit into a complete security solution.The security community is not learning from history lessons well-known in other sciences. Simply wishing for a “Science of Security” will not make it happen. Security researchers need to learn and adopt more scientific methodologies.

You might also like

Keywords