The core of this model is a softmax logistic function, which only included the following: a parameter that estimates any overall bias to respond fast or slow, an (unconstrained) ε parameter for uncertainty bonus, a softmax gain parameter, and an estimate of the value of the two actions. The latter was simulated either as the mean of the beta distribution

or a Q-value learned via reinforcement learning (RL) with an estimated learning rate. This categorical model identified a group of eight explore participants (ε > 0) that largely overlapped with the primary model (two of Selleck Trichostatin A 15 participants differed in assignment). Notably, the relative uncertainty effect in the eight explore participants from this categorical model yielded activation in dorsal RLPFC (XYZ = 24 50 18; 34 52 16; 44 42 28; p < 0.001 [FWE cluster level]), ventral RLPFC (XYZ = 36 56 −10; p < 0.005 [FWE cluster level]), and

SPL (XYZ = −8 −64 66; p < 0.001 [FWE cluster level]; Table S2). Again, there were no positive or negative correlations with relative uncertainty in RLPFC in the participants with negative ε. Thus, the effects of relative uncertainty in RLPFC were robust to these variations of the model. Moreover, in these models without a positive ε constraint, we did not find evidence that RLPFC tracks relative uncertainty in support of uncertainty aversion (i.e., participants with negative ε). However, this leaves open how to interpret negative epsilon in the nonexplore participants. As noted whatever above, one possibility is that participants tend to repeatedly select the same option independent from their values. When controlling

PF-01367338 for sticky choice in the categorical model, the majority of participants were best characterized by positive ε (11 or 13 out of 15 participants for beta or Q-learning variants, respectively). A likelihood ratio test confirmed that including an uncertainty exploration bonus provided a significantly better fit (and including penalization of extra parameters) across the group of explorers (defined from those in the standard model; p < 0.00001), but only marginally so in nonexplorers (p = 0.053; the test was significant across the whole group, p < 0.00001). In the Q version, the likelihood ratio test was again significant in the explorers, p = 0.00002, but not in the nonexplorers (p = 0.15; thus the slightly positive ε values did not contribute to model fit). This test was again significant across the entire group (p = 0.00005). As in prior models, the fitted ε parameter correlated with improvement in likelihood relative to a model without uncertainty driven exploration (r = 0.71, p = 0.003). Thus in these simplified models predicting categorical choice, only explorers showed a robust improvement in fit by incorporating relative uncertainty into the model, and a fit of negative epsilon seems largely explained by the tendency to perseverate independently of value.