> ## Documentation Index
> Fetch the complete documentation index at: https://wb-21fd5541-docs-3133.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Evaluation<R, E, M>

> TypeScript SDK reference

export const GitHubLink = ({url, compact = false}) => <a href={url} target="_blank" rel="noopener noreferrer" className={compact ? "source-link" : "github-source-link"}>
    {compact ? "View source" : <>
    <svg width="20" height="20" viewBox="0 0 24 24" fill="currentColor" xmlns="http://www.w3.org/2000/svg">
      <path d="M12 0C5.37 0 0 5.37 0 12c0 5.31 3.435 9.795 8.205 11.385.6.105.825-.255.825-.57 0-.285-.015-1.23-.015-2.235-3.015.555-3.795-.735-4.035-1.41-.135-.345-.72-1.41-1.23-1.695-.42-.225-1.02-.78-.015-.795.945-.015 1.62.87 1.845 1.23 1.08 1.815 2.805 1.305 3.495.99.105-.78.42-1.305.765-1.605-2.67-.3-5.46-1.335-5.46-5.925 0-1.305.465-2.385 1.23-3.225-.12-.3-.54-1.53.12-3.18 0 0 1.005-.315 3.3 1.23.96-.27 1.98-.405 3-.405s2.04.135 3 .405c2.295-1.56 3.3-1.23 3.3-1.23.66 1.65.24 2.88.12 3.18.765.84 1.23 1.905 1.23 3.225 0 4.605-2.805 5.625-5.475 5.925.435.375.81 1.095.81 2.22 0 1.605-.015 2.895-.015 3.3 0 .315.225.69.825.57A12.02 12.02 0 0024 12c0-6.63-5.37-12-12-12z" />
    </svg>
    GitHub source
      </>}
  </a>;

<GitHubLink url="https://github.com/wandb/weave/blob/7c9efdc9fe05ffcaf2430dd652ef9bc5b3ffdfaa/sdks/node/src/evaluation.ts#L145" />

Sets up an evaluation which includes a set of scorers and a dataset.

Calling evaluation.evaluate(model) will pass in rows form a dataset into a model matching
the names of the columns of the dataset to the argument names in model.predict.

Then it will call all of the scorers and save the results in weave.

## Example

```ts twoslash theme={null}
// @noErrors
// Collect your examples into a dataset
const dataset = new weave.Dataset({
  id: 'my-dataset',
  rows: [
    { question: 'What is the capital of France?', expected: 'Paris' },
    { question: 'Who wrote "To Kill a Mockingbird"?', expected: 'Harper Lee' },
    { question: 'What is the square root of 64?', expected: '8' },
  ],
});

// Define any custom scoring function
const scoringFunction = weave.op(function isEqual({ modelOutput, datasetRow }) {
  return modelOutput == datasetRow.expected;
});

// Define the function to evaluate
const model = weave.op(async function alwaysParisModel({ question }) {
  return 'Paris';
});

// Start evaluating
const evaluation = new weave.Evaluation({
  id: 'my-evaluation',
  dataset: dataset,
  scorers: [scoringFunction],
});

const results = await evaluation.evaluate({ model });
```

## Extends

* [`WeaveObject`](./weaveobject)

## Type Parameters

<ParamField path="R" type="extends DatasetRow" />

<ParamField path="E" type="extends DatasetRow" />

<ParamField path="M" />

## Constructor

> **new Evaluation**\<`R`, `E`, `M`>(`parameters`): `Evaluation`\<`R`, `E`, `M`>

### Parameters

<ParamField path="parameters" type="object" required>
  <Expandable title="properties" defaultOpen>
    <ParamField path="columnMapping" type="{[key: string]: keyof R; __savedRef?: keyof R; }">
      <Expandable title="properties" defaultOpen>
        <ParamField path="__savedRef" type="keyof R" />
      </Expandable>
    </ParamField>

    <ParamField path="dataset" type="Dataset<R>" required>
      See [`Dataset`](./dataset).
    </ParamField>

    <ParamField path="description" type="string" />

    <ParamField path="maxConcurrency" type="number" />

    <ParamField path="name" type="string" />

    <ParamField path="scorers" type="(Op<(...args) => any> | {…})[]" required>
      See [`Op`](../type-aliases/op).

      Full type: `(Op<(...args) => any> | { id: string; invoke: (...args) => any; })[]`
    </ParamField>
  </Expandable>
</ParamField>

### Returns

`Evaluation`\<`R`, `E`, `M`>

### Overrides

[`WeaveObject`](./weaveobject).[`constructor`](./weaveobject#constructor)

## Properties

### \_\_savedRef

> `optional` **\_\_savedRef**: [`ObjectRef`](./objectref) | `Promise`\<[`ObjectRef`](./objectref)>

#### Inherited from

[`WeaveObject`](./weaveobject).[`__savedRef`](./weaveobject#__savedref)

## Accessors

### description

#### Get Signature

> **get** **description**(): `string` | `undefined`

##### Returns

`string` | `undefined`

#### Inherited from

[`WeaveObject`](./weaveobject).[`description`](./weaveobject#description)

***

### name

#### Get Signature

> **get** **name**(): `string`

##### Returns

`string`

#### Inherited from

[`WeaveObject`](./weaveobject).[`name`](./weaveobject#name)

## Methods

### evaluate()

> **evaluate**(`__namedParameters`): `Promise`\<`Record`\<`string`, `any`>>

#### Parameters

<ParamField path="__namedParameters" type="object" required>
  <Expandable title="properties" defaultOpen>
    <ParamField path="maxConcurrency" type="number = 5" />

    <ParamField path="model" type="Op<(...args) => Promise<M>> | {…}" required>
      See [`Op`](../type-aliases/op).

      Full type: `Op<(...args) => Promise<M>> | { id: string; invoke: (...args) => Promise<M>; }`
    </ParamField>

    <ParamField path="nTrials" type="number = 1" />
  </Expandable>
</ParamField>

#### Returns

`Promise`\<`Record`\<`string`, `any`>>

***

### predictAndScore()

> **predictAndScore**(`__namedParameters`): `Promise`\<\{ `model_latency`: `number`; `model_output`: `any`; `model_success`: `boolean`; `scores`: \{\[`key`: `string`]: `any`; }; }>

#### Parameters

<ParamField path="__namedParameters" type="object" required>
  <Expandable title="properties" defaultOpen>
    <ParamField path="columnMapping" type="{[key: string]: keyof R; __savedRef?: keyof R; }">
      <Expandable title="properties" defaultOpen>
        <ParamField path="__savedRef" type="keyof R" />
      </Expandable>
    </ParamField>

    <ParamField path="example" type="R" required />

    <ParamField path="model" type="Op<(...args) => Promise<M>> | {…}" required>
      See [`Op`](../type-aliases/op).

      Full type: `Op<(...args) => Promise<M>> | { id: string; invoke: (...args) => Promise<M>; }`
    </ParamField>
  </Expandable>
</ParamField>

#### Returns

`Promise`\<\{ `model_latency`: `number`; `model_output`: `any`; `model_success`: `boolean`; `scores`: \{\[`key`: `string`]: `any`; }; }>

***

### saveAttrs()

> **saveAttrs**(): `object`

#### Returns

`object`

#### Inherited from

[`WeaveObject`](./weaveobject).[`saveAttrs`](./weaveobject#saveattrs)