STRUCTURED ACCESS FOR THIRD-PARTY RESEARCH ON FRONTIER AI MODELS: INVESTIGATING RESEARCHERS’ MODEL ACCESS REQUIREMENTS

The ability to sample from a model: this could include the ability to sample from the model in an automated manner, specifying the sampling algorithm and associated hyper-parameters, or access the probabilities and logits associated with the model’s outputs. [...] These are the algorithms that, given the logits of next tokens, selects the single token to appear next in the model’s output.6 Fine-tuning The second category of system access concerns the fine-tuning of models, that is, additional training, possibly on specific tasks, of a pretrained ‘base’ model. [...] Alternatively, the user could be presented with the logits or probabilities of each token generated in the output, or the logits or probabilities of the top-n most likely tokens at each position of the output. [...] (C.1) Parameters: The ability to view the learnt parameters of the model – that is, the numerical values that determine how information is processed by the model and thus the generated output.7 (C.2) Activations & Attention: The ability to view activations and attention patterns for a given input to the model. [...] This can be thought of as viewing the specific computations carried out to transform the user-provided input into the model-generated output.8 (C.3) Gradients: Gradients can be thought of as the updates made to a model’s parameters during training, in order to improve performance on the training objective.