Worker pool? #31

Activity

1. developit commented on May 29, 2018

   

   developit

   on May 29, 2018

   Last edited by developit

   Owner

   More actions

   This could probably be adapted (and simplified) to suit:
   https://github.com/proxy/gist.github.com/developit/65a2212731f6b00a8aaa55d70c594f5c

   FWIW, it's best to only invoke greenlet once per function - generally at definition time. Dynamically creating workers to do single tasks will always have a nontrivial performance overhead.
2. developit

   added

   questionFurther information is requestedFurther information is requested

   on May 29, 2018
3. developit commented on Oct 1, 2019

   

   developit

   on Oct 1, 2019

   Owner

   More actions

   Just going to clarify for folks finding this issue - Greenlet does re-use the worker thread for all calls, but only if you hoist the greenlet() definition so it's being called one time for the given async function:

   // BAD: this creates a new worker every time getPage() is called:
   export function getPage(url) {
     const getJson = greenlet(async url => {
       return await (await fetch(url)).json();
     });
     return getJson(url);
   }

   // GOOD: this uses the same worker for each call to getPage():
   const getJson = greenlet(async url => {
     return await (await fetch(url)).json();
   });
   export function getPage(url) {
     return getJson(url);
   }

   👍React with 👍1Reacted by Spencer Levitt
4. developit

   added

   has fix

   on Oct 1, 2019
5. imedadel commented on Apr 23, 2020

   

   imedadel

   on Apr 23, 2020

   More actions

   hey @developit, I wanted to offload some functions to a worker and I was thinking if this 👇 solution is good enough? (the alternative is using those postMessage thingies by hand, which I hate):

   let dispatch = greenlet(async (action) => {
     switch (action.type) {
       case 'fetchGH': {
         let url = `https://github.com/proxy/api.github.com/users/${action.username}`
         let res = await fetch(url)
         let profile = await res.json()
         return profile.name
       }
       case 'sayHi': {
         return "Hi " + action.name
       }
       default: {
         throw new Error(`Unknown action ${action.type}`)
       }
     }
   })
6. developit commented on Apr 23, 2020

   

   developit

   on Apr 23, 2020

   Owner

   More actions

   @imedadel that works, yup! There's also workerize, which is basically like Greenlet but with support for "exporting" multiple methods from the worker.

   ❤️React with ❤️1Reacted by Imed Adel
7. imedadel commented on Apr 24, 2020

   

   imedadel

   on Apr 24, 2020

   More actions

   @developit I love workerize(-loader)! But it didn't play well with Next.js, so I ended up using worker-plugin (which is also by you? :o) along with comlink.

   So, I guess, if anyone is trying to bundle workers in Next.js, use that combo, it works perfectly. Otherwise, stick to greenlet and/or workerize.

   (you should probably mark the last 3 comments as off-topic :) )
8. developit commented on Apr 24, 2020

   

   developit

   on Apr 24, 2020

   Owner

   More actions

   Oh I didn't mean the loader - there is a non-loader version of workerize that uses the same internals as Greenlet, works at runtime.
9. imedadel commented on Apr 24, 2020

   

   imedadel

   on Apr 24, 2020

   More actions

   I needed to import the AWS SDK, that's why it wasn't an option :(

   PS. If anyone needs to bundle web workers in Next.js, use Parcel (microbundle had a babel-related error) to build your workers to the /public folder. Then use comlink.

   Otherwise use greenlet/workerize.