Exam 3 readings

• explore: gathering of information (can be curse) exploit: using said information (never exploiting can be bad)

multi-armed bandit problem

the bandit problem embodies an essential form a conflict evident in all human actions

the interval overwich we plan to enjoy = the strategy

• choose an arm at random and keep pulling as long as it pays off • of other doesn’t pay off after particular turn, switch no notion of interval over whcih you are optimizing

for problem with cases where we know exactly how many options/opportunities we have in total - start: imagining the final pull + consider which slot machine to choose given all the possibilities of outcomes based on previous decisions - turn to second-last opportunity, then the previous all the way to the start

conceived the goal as maximizing payoffs not for a fixed interval of time, but for a futrue that is endless yet discounted aka the dynamic allocation index

• value assigned + payoffs decreases geometrically (each restaurant visit you make is a constant fraction of the last) • always play the arm with the highest index values for upto 9 success + failures, assumed that a payoff on our next pull is worth 90% of a pay off now integrates the win/stay, lose/switch principle

• 0-0 is more attractive (higher index) than a machine you knwo pays 7 out of 10 times • untested > experienced primarily because we know less (unknown is good)

not on the fly since it takes way too long

focus on it if you are not well characterized by geometric discounting (can be highly motivating e.g., Jeff Bezos)

• assuming you are omniscient, your total amount of regret will never stop increasing • regret increases at a slower rate if you pick the best strategy (rate will slow over time as you learn to make better decisions) • the minimum possible regret is regret that increases at a logarithmic rate with every pull of the handle (make as many mistakes in first 10 pulls as in the following 90)

• error bars (uncertainty in the measurement) range of plausible values that the quality being measured could actually have (AKA confidence interval) • as we gain more data, confidence interval goes down (= more accurate data)

• the option with the highest confidence index • doesn’t care about which arm has performed best so far but which will/could perform best in the future (potential for greatness)

• optimism! (optimism in the face of uncertainty)

• company drafts multiple different designs and each user is shown one of them (measure engagement) • explore/exploit problem • deeply embedded pat of how business + politics work

• hat containing a ball for each of the 2 treatments being studied. • draw a ball, treat patient, put extra ball for the treatment that worked –> increasing it’s chance of being picked slightly unethical but works!

• tend to overexplore when math indicated that they really didnt need to (stared at the bulb too much) • used the untried airline too little when it was good + too much when it was bad

• longer period of dependance (as a child) which solves the exploration/exploitation tradeoff • you just explore and the pay offs are taken care of by your guardians

• elders have fewer relationships by choice • pruning relationships + focusing attention on core relationships (most meaningful) • about where people perceive themselves to be on the interval relevant to their decision (sensitive about how much time you have left) • old age is time of exploitation (phenomena of aging)

being let down on most occasions - shifting the bulk of one’s attention to one’s favourite things should increase the quality of life

• means for an individual to acquire superior knowledge and make better decisions • by performing this intellectual function, elevates humans above all other animals

misunderstood in the past which led to wrong use. - selected traits have at least one of their effects tht has contributed to the reproductive success of their carriers and thereby to their own propagation - adaptation: traits that owe their propagation to their function (e.g., regular contractions of the heart - side effect: the noise a heart makes due to the contractions

commonsense rather than biological notion (that was the problem)