# Custom trainer evaluation function

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustom-trainer-evaluation-function%2F19394%3E%5C
**Category:** Intermediate\
**Created:** [June 20, 2022, 7:21pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustom-trainer-evaluation-function%2F19394 "2022-06-20T19:21:14Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![wz1232](https://avatars.discourse-cdn.com/v4/letter/w/ebca7d/32.png) [@wz1232](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fwz1232)%5C
**Post date:** [June 20, 2022, 7:21pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustom-trainer-evaluation-function%2F19394%2F1 "2022-06-20T19:21:14Z")

</div>

Hi, so I am trying to override the Trainer evaluate() function with my own method. It runs, but the problem is that when I use multi-GPUs (8 in my case), it seems to split the eval dataset across the 8 GPUs but only report the metric calculated for the eval subset on 1 GPU.

My boilerplate code is shown in here: [scratch/trainer\_eval.py at master · kevinghst/scratch · GitHub](https://github.com/kevinghst/scratch/blob/master/trainer_eval.py)

Can someone please help take a look and identify the underlying reason for this behavior?

Thanks,  
Kevin