(The below text version of the notes is for search purposes and convenience. See the PDF version for proper formatting such as bold, italics, etc., and graphics where applicable. Copyright: 2022 Retraice, Inc.)
Re15: Trust and Sources
Retraice^1
On the neglected basis of all knowledge and belief.
Air date: Wednesday, 25th Nov. 2020, 1:30 PM Pacific/US.
Detection and reports
Perhaps the only difference between something you perceive directly and something somebody tells you is how much you trust it.
Instead of talking about `sources', we can talk about detection and reports. Cf. Russell on knowledge by acquaintance (detection) and knowledge by description (reports).^2
The mirrors of Rjukan, Norway
They don't get enough sunlight at that latitude and near those mountains, so they reflect sunlight into the town using mirrors.^3
If you believe this, it is because reports and trust have convinced you (unless you've been to Rjukan).
Fakes, tech and incentives
While new technologies (e.g. deep-faking) do make fake reports (images, audio, videos) more possible, only incentives can make fakes probable. But as technology gets better, the requisite level of faking-incentive goes down.
Do you trust everything you perceive?
Hallucinations can be profound or barely noticeable.^4 Michael Shermer thought he was abducted by aliens but later concluded he was just riding too much bike.^5 What changed was the trust--he went from trusting earlier detection to trusting later detection and reports from others.
Trust is a mess
There's no clear, widely-known way to think about trust and sources. Adding `detection and reports' doesn't help.
Recall Cynthia Grabo^6 on deception.
A hundred years ago, Walter Lippmann wrestled with this problem.^7 It was essentially the same problem back then that it is today. Edward Bernays also worked on it.^8 Noam Chomsky had lots to say about both of them.^9 See also Ken Thompson from the programming point of view.^10
Even if you have a good system for deciding trust problems^11 , an adversary who knows you're using that system will probably find weaknesses in it and cause you to mistrust. This gets dark.^12 And it's also what computer hacking is about.^13
The concept of trust is scattered throughout the security-related and other literatures, as:
o essential to military hardware^14 o catalyst of intelligence work^15 o prerequisite to espionage^16 o leading to a deception paradox'^17 o the central problem of counterespionage^18 o most dangerous in the form ofinsiders'^19 o the central problem of Internet security^20 o obviating the need for large-scale surveillance and enabling large-scale long-term cooperation^21 o gained by contracts, credit scores and references^22 o gained by deception^23 o gained by self-deception^24 o essential to rationality^25 o essential to science and its division of labor^26 o burden on the trustee^27 o a deciding factor in whether to use a neural network^28 o a dividing line between AI developers and AI users^29 o essential in sales^30 o essential in writing^31
etc. But where is the clear, widely-known way to think about it?
References
Adee, S. (2008). The hunt for the kill switch. IEEE Spectrum, 45(5), 34-39. May 2008.
Primary source and search:
https://ieeexplore.ieee.org/document/4505310 Retrieved 27th Feb. 2022.
https://www.google.com/search?q=%22hunt+for+the+kill+switch%22+pdf
Bernays, E. (1923). Crystallizing Public Opinion. Boni and Liveright (Reprint Ig Publishing). ISBN: 978-1935439264. Searches:
https://www.amazon.com/s?k=978-1935439264
https://www.google.com/search?q=isbn+978-1935439264
https://lccn.loc.gov/2011025346
Bernays, E. (1928). Propaganda. H. Liveright (Reprint by Ig Publishing). ISBN: 978-0970312594. Searches:
https://www.amazon.com/s?k=978-0970312594
https://www.google.com/search?q=isbn+978-0970312594
https://lccn.loc.gov/2004016015
Bernays, E. (1952). Public Relations. University of Oklahoma Press. ISBN: 978-0806114576. Searches:
https://www.amazon.com/s?k=978-0806114576
https://www.google.com/search?q=isbn+978-0806114576
https://lccn.loc.gov/52001707
Broughton, P. D. (2012). The Art of the Sale: Learning from the Masters About the Business of Life. Penguin. ISBN: 978-1594203329. Searches:
https://www.amazon.com/s?k=978-1594203329
https://www.google.com/search?q=isbn+978-1594203329
https://lccn.loc.gov/2011040209
Chomsky, N. (1989). Necessary Illusions: Thought Control in Democratic Societies. South End Press. ISBN: 0896083667. Searches:
https://www.amazon.com/s?k=0896083667
https://www.google.com/search?q=isbn+0896083667
https://lccn.loc.gov/89006074
Chomsky, N. (2017). Requiem for the American Dream: The 10 Principles of Concentration of Wealth & Power. Seven Stories Press. ISBN: 978-1609807368. Searches:
https://www.amazon.com/s?k=978-1609807368
https://www.google.com/search?q=isbn+978-1609807368
https://lccn.loc.gov/2016054121
de Becker, G. (1997). The Gift of Fear: And Other Survival Signals That Protect Us from Violence. Dell / Random House. ISBN: 0440508835. Searches:
https://www.amazon.com/s?k=0440508835
https://www.google.com/search?q=isbn+0440508835
https://lccn.loc.gov/96051051
Dulles, A. (2016). The Craft of Intelligence. Lyons Press / Rowman & Littlefield. First published 1963. This edition copyright Joan Buresch Talley, daughter of Dulles. ISBN: 978-1493018796. Searches:
https://www.amazon.com/s?k=978-1493018796
https://www.google.com/search?q=isbn+978-1493018796
https://lccn.loc.gov/2016017105
Different editions available at:
https://archive.org/search.php?query=The%20Craft%20of%20Intelligence
Erickson, J. (2008). Hacking: The Art of Exploitation. No Starch Press, 2nd ed. ISBN: 978-1593271442. Searches:
https://www.amazon.com/s?k=978-1593271442
https://www.google.com/search?q=isbn+978-1593271442
https://lccn.loc.gov/2007042910
Gopnik, A., Meltzoff, A. N., & Kuhl, P. K. (1999). The Scientist in the Crib: What Early Learning Tells Us About the Mind. Perennial / HarperCollins. ISBN: 0688159885. Searches:
https://www.amazon.com/s?k=0688159885
https://www.google.com/search?q=isbn+0688159885
https://lccn.loc.gov/99024247
Grabo, C. M. (2002). Anticipating Surprise: Analysis for Strategic Warning. Center for Strategic Intelligence Research. ISBN: 0965619567
https://www.ni-u.edu/ni_press/pdf/Anticipating_Surprise_Analysis.pdf Retrieved 7th Sep. 2020.
Hawley, K. (2012). Trust: A Very Short Introduction. Oxford University Press. ISBN: 9780199697342. Searches:
https://www.amazon.com/s?k=9780199697342
https://www.google.com/search?q=isbn+9780199697342
https://lccn.loc.gov/2012406649
Herman, E. S., & Chomsky, N. (1988). Manufacturing Consent: The Political Economy of the Mass Media. Pantheon. ISBN: 0679720340. Searches:
https://www.amazon.com/s?k=0679720340
https://www.google.com/search?q=isbn+0679720340
https://lccn.loc.gov/88042614
Kidder, T., & Todd, R. (2013). Good Prose. Random House Publishing Group, kindle edition ed. ISBN: 9780679604723. Searches:
https://www.amazon.com/s?k=9780679604723
https://www.google.com/search?q=isbn+9780679604723
https://lccn.loc.gov/2012021165
Lippmann, W. (1920). Liberty and the News. Harcourt, Brace and Howe (Leopold Reprint). No ISBN. eBook and searches:
https://books.google.com/books?id=Df-SzcLRcAIC Retrieved 24th Feb. 2022.
https://www.amazon.com/s?k=Liberty+and+the+News+Lippmann
https://www.google.com/search?q=liberty+and+the+news+lippmann
https://lccn.loc.gov/20004814
Oleson, P. C. (Ed.) (2016). AFIO's Guide to the Study of Intelligence. Association of Former Intelligence Officers, 1st ed. Citations are of the pbk. edition, ISBN: 978-0997527308. PDF edition available at:
https://www.afio.com/40_guide.htm Retrieved 10th Sep. 2020.
Peterson, M. (2017). An Introduction to Decision Theory. Cambridge University Press, 2nd ed. ISBN: 9781316606209. Searches:
https://www.amazon.com/s?k=9781316606209
https://www.google.com/search?q=isbn+9781316606209
https://lccn.loc.gov/2016057387
Pinker, S. (2011). The Better Angels of Our Nature: Why Violence Has Declined. Penguin Publishing Group. ISBN: 978-0143122012. Searches:
https://www.amazon.com/s?k=978-0143122012
https://www.google.com/search?q=isbn+978-0143122012
https://lccn.loc.gov/2011015201
Retraice (2020/09/07). Re1: Three Kinds of Intelligence. retraice.com.
https://www.retraice.com/segments/re1 Retrieved 22nd Sep. 2020.
Russell, B. (1912). The Problems of Philosophy. Dover (Reprinted 1999). ISBN: 0486406741. Searches:
https://www.amazon.com/s?k=0486406741
https://www.google.com/search?q=isbn+0486406741
https://lccn.loc.gov/98033214
Russell, S., & Norvig, P. (2020). Artificial Intelligence: A Modern Approach. Pearson, 4th ed. ISBN: 978-0134610993. Searches:
https://www.amazon.com/s?k=978-0134610993
https://www.google.com/search?q=isbn+978-0134610993
https://lccn.loc.gov/2019047498
Sacks, O. (2012). Hallucinations. Vintage Books (Reprinted 2013). ISBN: 978-0307947437. Searches:
https://www.amazon.com/s?k=978-0307947437
https://www.google.com/search?q=isbn+978-0307947437
https://lccn.loc.gov/2012002877
Salter, A. (2003). Predators. Basic Books. ISBN: 978-0465071732. Searches:
https://www.amazon.com/s?k=978-0465071739
https://www.google.com/search?q=isbn+978-0465071739
https://lccn.loc.gov/2002015846
Schneier, B. (2003). Beyond Fear: Thinking Sensibly About Security in an Uncertain World. Copernicus Books. ISBN: 0387026207. Searches:
https://www.amazon.com/s?k=0387026207
https://www.google.com/search?q=isbn+0387026207
https://lccn.loc.gov/2003051488
Similar edition available at:
https://archive.org/details/beyondfearthinki00schn_0
Schneier, B. (2015). Data and Goliath: The Hidden Battles to Collect Your Data and Control Your World. W. W. Norton. ISBN: 9780393244816. Searches:
https://www.amazon.com/s?k=9780393244816
https://www.google.com/search?q=isbn+9780393244816
https://lccn.loc.gov/2014048365
Shermer, M. (2005). Abducted! Imaginary traumas are as terrifying as the real thing. Scientific American. Feb. 2005.
https://michaelshermer.com/sciam-columns/abducted/ Retrieved 25th Nov. 2020.
https://www.scientificamerican.com/article/abducted/ Retrieved 28th Feb. 2022.
Simler, K., & Hanson, R. (2018). The Elephant in the Brain: Hidden Motives in Everyday Life. Oxford University Press. ISBN: 9780190495992. Searches:
https://www.amazon.com/s?k=9780190495992
https://www.google.com/search?q=isbn+9780190495992
https://lccn.loc.gov/2017004296
Thompson, K. (1984). Reflections on trusting trust. Communications of the ACM, 27(8), 761-763. Aug. 1984.
https://doi.org/10.1145/358198.358210 Also available at:
https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ReflectionsonTrustingTrust.pdf Retrieved 4th Dec. 2020.
Footnotes
^1 https://www.retraice.com/retraice
^2 Russell (1912) p. 31 ff.
^3 https://www.google.com/search?q=mirrors+Rjukan
^4 Sacks (2012).
^5 Shermer (2005). See also Sacks (2012) p. 42 ff. on Shermer's case.
^6 Grabo (2002) p. 119 ff. See also Retraice (2020/09/07).
^7 See Lippmann (1920) e.g. pp. 37-38, where he talks about second, third and forth hand knowledge, the problem of seeing for oneself, and the unreliability of such methods.
^8 Bernays (1928) pp. 54-55 where trust is implied by his discussions of authority' andloyalty'; Bernays (1923) p. 191 on "the most valuable thing" being "faith and trust"; Bernays (1952) p. 112 on PR and suspicion of businessmen.
^9 Among many examples, see Herman & Chomsky (1988) p. xi, p. 332; Chomsky (1989) pp. 16-17; Chomsky (2017) pp. 124-125.
^10 Thompson (1984) on trusting statements, programs and people.
^11 Is this a hint, the word `deciding'? Are all trust problems really decision problems?
^12 Salter (2003) p. 103; Pinker (2011) p. 484.
^13 Erickson (2008) p. 1. The implication is this: If we trust a system that we don't fully understand we become vulnerable to events and actions by others that we cannot anticipate.
^14 Oleson (2016) p. 275 and Adee (2008).
^15 Oleson (2016) p. 231.
^16 Oleson (2016) p. 470.
^17 Oleson (2016) p. 404.
^18 Dulles (2016) pp. 143-144.
^19 Schneier (2003) pp. 62-63.
^20 Schneier (2015) pp. 181-183.
^21 Simler & Hanson (2018) p. 271.
^22 Simler & Hanson (2018) p. 271.
^23 de Becker (1997) p. 59.
^24 Simler & Hanson (2018) pp. 81-83.
^25 Peterson (2017) p. 10 and pp. 266-267.
^26 Gopnik et al. (1999) p. 160.
^27 Hawley (2012) p. 14.
^28 Russell & Norvig (2020) pp. 711-712.
^29 Russell & Norvig (2020) p. 996 ff.
^30 Broughton (2012) p. 291 (index).
^31 Kidder & Todd (2013) location 129.