Shortround's Space

Early exits in javascript generators

I was debugging a generator exiting early, seemingly without passing any of my final logging statements:

async function *getNextSponsoredCarouselProduct(
    logger: ILogger | undefined,
    requestBids: RequestBidsFn,
    generateProductCardResponse: GenerateSponsoredProductCardResponseFn,
    request: IBidRequest
): AsyncGenerator<IBuiltProductCardBidResponse> {
    const log = Ctx(logger)?.with('getNextSponsoredCarouselProduct');
    try {
        log?.info('Entering sponsored carousel product generator');
        while(true) {
            log?.info('Getting next sponsored carousel products', request);
            const response = await requestBids(request);
            if(!response.ok) {
                log?.info('No response from bidder', response.error);
                break;
            }

            const bids = response.value.seatbid?.[0]?.bid;
            if(bids === undefined) {
                log?.info('Got no bids in response!');
                break;
            }

            const generator = generateProductCardResponse(bids);
            for await(const product of generator) {
                log?.debug('Getting next product from existing bidder response', product);
                yield product;
            }
            log?.debug('Out of bids from previous response. Requesting new bid response');
        }
    } catch(err) {
        log?.error('product carousel generator failed! Got', err);
    }
    log?.warn('Exiting carousel product generator');   

}

I was seeing Getting next product from existing bidder response, and that was it! Further consumers of my generator would get nothing, and I never saw Exiting carousel product generator. Weird!

I changed the final log message to a finally:

} finally {
    log?.warn('Exiting carousel product generator');   
}

and finally, I saw the log! So something WAS destroying my generator, it wasn't an exception or anything. At first I suspected some kind of garbage collection - perhaps my generator reference was being destroyed after its first usage?

    const generator = carousel.getProduct();
    rightArrow.addEventListener('click', () => {
        handleNextPage(log,
            generator // Was the reference being lost here somehow?
        );
    });

But that wasn't it. I changed generator to window.generator to make sure the reference was globally stored, just for testing. Same problem

It turns out it was a little helper I wrote:

export const ConsumeNAsync = async <T>(generator: AsyncGenerator<T>, n: number): Promise<T[]> => {
    const consumed: T[] = [];
    for await (const value of generator) {
        consumed.push(value);
        if(consumed.length >= n) {
            break;
        }
    }
    return consumed;
}

The way I read this code is: "consume from the generator until I tell you to stop. When the array has >= n items, stop".

But what is secretly going on under the hood is when you break out of a for-await loop with a generator, it kills the generator.

This is one of those things that, I'm sure if you explained the "why", would make sense in abstract. But personally, this looks like a landmine to me. I would not have expected the generator to be destroyed upon breaking out of the loop.

The corrected code is:

export const ConsumeNAsync = async <T>(generator: AsyncGenerator<T>, n: number): Promise<T[]> => {
    const consumed: T[] = [];
    while(consumed.length < n) {
        const result = await generator.next();
        if(result.done) {
            break;
        }
        consumed.push(result.value);
    }
    return consumed;
}

This will preserve the state of the generator beyond the while-loop

Thoughts? Leave a comment