I was debugging a generator exiting early, seemingly without passing any of my final logging statements:
async function *getNextSponsoredCarouselProduct( logger: ILogger | undefined, requestBids: RequestBidsFn, generateProductCardResponse: GenerateSponsoredProductCardResponseFn, request: IBidRequest ): AsyncGenerator<IBuiltProductCardBidResponse> { const log = Ctx(logger)?.with('getNextSponsoredCarouselProduct'); try { log?.info('Entering sponsored carousel product generator'); while(true) { log?.info('Getting next sponsored carousel products', request); const response = await requestBids(request); if(!response.ok) { log?.info('No response from bidder', response.error); break; } const bids = response.value.seatbid?.[0]?.bid; if(bids === undefined) { log?.info('Got no bids in response!'); break; } const generator = generateProductCardResponse(bids); for await(const product of generator) { log?.debug('Getting next product from existing bidder response', product); yield product; } log?.debug('Out of bids from previous response. Requesting new bid response'); } } catch(err) { log?.error('product carousel generator failed! Got', err); } log?.warn('Exiting carousel product generator'); }
I was seeing Getting next product from existing bidder response
, and that was it! Further consumers of my generator would get nothing, and I never saw Exiting carousel product generator
. Weird!
I changed the final log message to a finally
:
} finally { log?.warn('Exiting carousel product generator'); }
and finally, I saw the log! So something WAS destroying my generator, it wasn't an exception or anything. At first I suspected some kind of garbage collection - perhaps my generator reference was being destroyed after its first usage?
const generator = carousel.getProduct(); rightArrow.addEventListener('click', () => { handleNextPage(log, generator // Was the reference being lost here somehow? ); });
But that wasn't it. I changed generator
to window.generator
to make sure the reference was globally stored, just for testing. Same problem
It turns out it was a little helper I wrote:
export const ConsumeNAsync = async <T>(generator: AsyncGenerator<T>, n: number): Promise<T[]> => { const consumed: T[] = []; for await (const value of generator) { consumed.push(value); if(consumed.length >= n) { break; } } return consumed; }
The way I read this code is: "consume from the generator until I tell you to stop. When the array has >= n items, stop".
But what is secretly going on under the hood is when you break out of a for-await loop with a generator, it kills the generator.
This is one of those things that, I'm sure if you explained the "why", would make sense in abstract. But personally, this looks like a landmine to me. I would not have expected the generator to be destroyed upon breaking out of the loop.
The corrected code is:
export const ConsumeNAsync = async <T>(generator: AsyncGenerator<T>, n: number): Promise<T[]> => { const consumed: T[] = []; while(consumed.length < n) { const result = await generator.next(); if(result.done) { break; } consumed.push(result.value); } return consumed; }
This will preserve the state of the generator beyond the while-loop