Quotessence
Home / Authors / Nick Bostrom Books
Nick Bostrom

Nick Bostrom Books

Philosopher

Related Quotes

“Human individuals and human organizations typically have preferences over resources that are not well represented by an "unbounded aggregative utility function". A human will typically not wager all her capital for a fifty-fifty chance of doubling it. A state will typically not risk losing all its territory for a ten percent chance of a tenfold expansion. [T]he same need not hold for AIs. An AI might therefore be more likely to pursue a risky course of action that has some chance of giving it control of the world.”

Artificial IntelligenceRisk AversionSuperintelligenceDecision TheoryArtificial SuperintelligenceSingletonExpected UtilityNon Maximizing Decision RulesUtility Function
Book:Superintelligence: Paths, Dangers, Strategies
Create image

“Think of a "discovery" as an act that moves the arrival of information from a later point in time to an earlier time. The discovery's value does not equal the value of the information discovered but rather the value of having the information available earlier than it otherwise would have been. A scientist or a mathematician may show great skill by being the first to find a solution that has eluded many others; yet if the problem would soon have been solved anyway, then the work probably has not much benefited the world [unless having a solution even slightly sooner is immensely valuable or enables further important and urgent work].”

ScienceDiscoveryScientific DiscoveryTime Value Of MoneyTime ValueScientific WorkTime PreferenceCounterfactualCounterfactual ImpactCounterfactualsHigh Impact Science
Book:Superintelligence: Paths, Dangers, Strategies
Create image

“[D]umb evolutionary processes have dramatically amplified the intelligence in the human lineage even compared with our close relatives the great apes and our own humanoid ancestors; and there is no reason to suppose Homo sapiens to have reached the apex of cognitive effectiveness attainable in a biological system.”

EvolutionIntelligenceBiologyHuman BrainNatural IntelligenceCognitive EvolutionHuman Exceptionalism
Book:Superintelligence: Paths, Dangers, Strategies
Create image

“Our approach to existential risks cannot be one of trial-and-error. There is no opportunity to learn from errors. The reactive approach - see what happens, limit damages, and learn from experience - is unworkable. Rather, we must take a proactive approach. This requires foresight to anticipate new types of threats and a willingness to take decisive preventive action and to bear the costs (moral and economic) of such actions.”

HappensActionOpportunityMoralRiskEconomicTypeBearsCostLimitsApproachThreatErrorsTrialsDamageWillingnessExistentialAnticipateForesightProactiveLearning ExperienceTrial And ErrorOpportunities To Learn
Author:Nick Bostrom
Create image

“The cognitive functioning of a human brain depends on a delicate orchestration of many factors, especially during the critical stages of embryo development-and it is much more likely that this self-organizing structure, to be enhanced, needs to be carefully balanced, tuned, and cultivated rather than simply flooded with some extraneous potion.”

NeedsHumansSelfBrainStageDependsDevelopmentStructureCriticalFactorsDelicateBalancedCognitiveHuman BrainEmbryosOrchestration
Book:Superintelligence: Paths, Dangers, Strategies
Create image

“How can we trace out the links between actions that people take today and really long-term outcomes for humanity - outcomes that stretch out indefinitely into the future? I call this effort macrostrategy - that is, to think about the really big strategic situation for having a positive impact on the long-term future. There's the butterfly effect: A small change in an initial condition could have arbitrarily large consequences.”

PeopleThinkingTodayActionHumanityEffortSituationConsequenceButterflyReally LongPositive Impact
Author:Nick Bostrom
Create image

“In the next century, we will be inventing radical new technologies - machine intelligence, perhaps nanotech, great advances in synthetic biology and other things we haven't even thought of yet. And those new powers will unlock wonderful opportunities, but they might also bring with them certain risks. And we have no track record of surviving those risks. So if there are big existential risks, I think they are going to come from our own activities and mostly from our own inventiveness and creativity.”

ThinkingOpportunityCreativityTechnologyWonderfulRiskTrackRadicalBiologyExistentialSurvivingNew TechnologyInventiveness
Author:Nick Bostrom
Create image

“The greatest existential risks over the coming decades or century arise from certain, anticipated technological breakthroughs that we might make in particular, machine super intelligence, nanotechnology and synthetic biology. Each of these has an enormous potential for improving the human condition by helping cure disease, poverty, etc. But one could imagine them being misused, used to create powerful weapon systems, or even some kind of accidental destructive scenario, where we suddenly are in possession of some technology that's far more powerful than we are able to control or use wisely.”

KindHelpingPowerfulPovertyTechnologyImagineRiskPossessionBiologyHuman ConditionExistentialTechnologicalBreakthroughMisused
Author:Nick Bostrom
Create image

“Far from being the smartest possible biological species, we are probably better thought of as the stupidest possible biological species capable of starting a technological civilization - a niche we filled because we got there first, not because we are in any sense optimally adapted to it.”

FirstsCivilizationCapableFilledSpeciesStartingTechnologicalNicheAdaptedStupidest
Book:Superintelligence: Paths, Dangers, Strategies
Create image

“It’s unlikely that any of those natural hazards will do us in within the next 100 years if we’ve already survived 100,000. By contrast, we are introducing, through human activity, entirely new types of dangers by developing powerful new technologies. We have no record of surviving those.”

IfsYearsHumansNextNaturalPowerfulTechnologyRecordsDangerTypeActivityDevelopingContrastIntroducingSurvivedUnlikelySurvivingHazardsNew TechnologyHuman Activity
Author:Nick Bostrom
Create image