From Generator to Async function

Preface

Speaking of asynchronous functions, can't help but think of [Wind.js](/articles/向 windjs 致敬-node 异步流程控制 4/), and Old Zhao's foresight:

Wind.js is absolutely an innovation in JavaScript asynchronous programming field,可谓 unprecedented. Some friends commented that "before seeing Wind.js, really thought this was impossible to implement", because Wind.js actually "patched" JavaScript language in the form of a library, and it's precisely this reason that allows JavaScript asynchronous programming experience to achieve a qualitative leap. ——July 2012

ES2017's async&await came through promise, generator all the way, while Wind saw this day as early as 6 years ago, and implemented the vision in advance

1. yield and await

Why say Async function comes from [Promise](/articles/完全理解 promise/), [Generator](/articles/generator（生成器）-es6 笔记 2/) all the way?

Because asynchronous functions have intricate relationships with Generator feature, for example, semantically both have pause meaning:

yield: give way, take a break catch breath
await: wait a minute

First compare a simplest scenario:

// generator
function* gen() {
  console.log('Do step 1');
  yield 'Until step1 completed';
  console.log('Do step 2');
}
let iter = gen();
iter.next();
iter.next();

// async function
async function f() {
  console.log('Do step 1');
  await 'Until step1 completed';
  console.log('Do step 2');
}
f();

Both code structures are similar, and output is also similar (executed separately as two examples):

// generator
Do step 1
Do step 2
{value: undefined, done: true}

// async function
Do step 1
Do step 2
Promise?{<resolved>: undefined}

2. Where's the Pause?

Generators can let execution flow "catch breath", can let things that can't stop pause, can be used to refactor loops, can control infinite sequences, can wrap iterators... many benefits

(From [generator_ ES6 Notes 2](/articles/generator（生成器）-es6 笔记 2/#articleHeader9))

But in above example seems didn't see pause effect, let's add some log, make everything more obvious:

// generator
function* gen() {
  console.log('Do step 1');
  yield 'Until step1 completed';
  console.log('Do step 2');
}
let iter = gen();
iter.next();
console.log('generator smokes a cigarette');
iter.next();

// async function
async function f() {
  console.log('Do step 1');
  await 'Until step1 completed';
  console.log('Do step 2');
}
f();
console.log('async function smokes a cigarette');

This time don't focus on respective return values (already saw above), execute together, output result as follows:

Do step 1
generator smokes a cigarette
Do step 2
Do step 1
async function smokes a cigarette
Do step 2

Output has no difference, but log('xxx smokes a cigarette') position has difference

Actual difference lies in, Generator's execution process in above example is purely synchronous, while async function's execution process contains asynchronous parts, described with Generator, equivalent to:

// generator pretending async function
function* gen() {
  console.log('Do step 1');
  yield Promise.resolve('Until step1 completed');
  console.log('Do step 2');
}
let iter = gen();
let step1 = iter.next();
step1.value.then(iter.next.bind(iter));
console.log('generator pretending async function smokes a cigarette');

// Output result
Do step 1
generator pretending async function smokes a cigarette
Do step 2

3. Closer, Even Closer

Further, very easy to use Generator to implement Async function feature:

function asyncFunction(gen, ...args) {
  return new Promise((resolve, reject) => {
    resolve(safeNext(gen(...args)));
  });
}

function safeNext(iter, last) {
  let step;
  try {
    step = iter.next(last);
  } catch(ex) {
    step = iter.throw(ex);
  }

  return Promise.resolve(step.value)
    .catch(ex => iter.throw(ex).value)
    .then(result => step.done ? result : safeNext(iter, result))
}

P.S. Github repo address ayqy/asyncFunction

Try it out:

asyncFunction(function* (){
  console.log('Do step 1');
  // Wait 100ms
  let x = yield new Promise((resolve, reject) => {
    setTimeout(resolve.bind(null, 1), 100);
  });
  // 100ms later
  console.log(`Step1 completed, got ${x}`);
  try {
    throw ++x;
  } catch(ex) {
    x = -1;
  }
  console.log(`x = ${x}`);
  x = yield x * 2;
  console.log(`All steps passed, got ${x}`);
  return x;
}).then(result => {
  console.log(`Final result ${result}`);
});
let intervalId = setInterval(console.log.bind(console, 'tick'), 10);
setTimeout(() => {
  clearInterval(intervalId);
}, 100);

Will get similar output:

Do step 1
3
⑨tick
Step1 completed, got 1
x = -1
All steps passed, got -2
Final result -2
tick

Among them second line's 3 is setTimeout return value (therefore asyncFunction only first segment is synchronously executed), third line outputs 9 times 'tick' indicates passed 90+ ms, at this point Wait 100ms ended, then execute remaining parts until end

Additionally, there's a hard to notice detail is, in this example remaining part's execution won't be interrupted by interval callback (even if interval is extremely short), for example:

asyncFunction(function* (){
  setTimeout(console.log.bind(console, '#0'), 0)
  console.log('Do step 1');
  // Wait 100ms
  let x = yield new Promise((resolve, reject) => {
    setTimeout(resolve.bind(null, 1), 100);
  });
  setTimeout(console.log.bind(console, '#1'), 0)
  // 100ms later
  console.log(`Step1 completed, got ${x}`);
  setTimeout(console.log.bind(console, '#2'), 0)
  try {
    throw ++x;
  } catch(ex) {
    x = -1;
  }
  setTimeout(console.log.bind(console, '#3'), 0)
  console.log(`x = ${x}`);
  x = yield x * 2;
  setTimeout(console.log.bind(console, '#4'), 0)
  console.log(`All steps passed, got ${x}`);
  return x;
}).then(result => {
  console.log(`Final result ${result}`);
});

Output result is:

Do step 1
Promise?{<pending>}
#0
Step1 completed, got 1
x = -1
All steps passed, got -2
Final result -2
#1
#2
#3
#4

#1, 2, 3, 4 output last, this is related to task type, specifically see macrotask vs microtask

Compare with official async function:

(async function(){
  setTimeout(console.log.bind(console, '#0'), 0)
  console.log('Do step 1');
  // Wait 100ms
  let x = await new Promise((resolve, reject) => {
    setTimeout(resolve.bind(null, 1), 100);
  });
  setTimeout(console.log.bind(console, '#1'), 0)
  // 100ms later
  console.log(`Step1 completed, got ${x}`);
  setTimeout(console.log.bind(console, '#2'), 0)
  try {
    throw ++x;
  } catch(ex) {
    x = -1;
  }
  setTimeout(console.log.bind(console, '#3'), 0)
  console.log(`x = ${x}`);
  x = await x * 2;
  setTimeout(console.log.bind(console, '#4'), 0)
  console.log(`All steps passed, got ${x}`);
  return x;
})().then(result => {
  console.log(`Final result ${result}`);
});

Output completely consistent:

Do step 1
Promise?{<pending>}
#0
Step1 completed, got 1
x = -1
All steps passed, got -2
Final result -2
#1
#2
#3
#4

4. Syntax Sugar?

Basic syntax form as follows:

async function name([param[, param[, ... param]]]) {
  statements
}

Need to know 2 points:

await keyword can only appear inside Async function, otherwise error
Async function's return value is Promise

Actually, async function has total 4 forms:

Function declaration: async function foo() {}
Function expression: const foo = async function () {};
Method definition: let obj = { async foo() {} }
Arrow function: const foo = async () => {};

For example:

async function fetchJson(url) {
  try {
    console.log('Starting fetch');
    let request = await fetch(url);
    let text = await request.text();
    return JSON.parse(text);
  } catch(error) {
    console.error(error);
  }
}

// test
fetchJson('https://unpkg.com/emoutils/package.json')
  .then(json => console.log(json));
console.log('Fetching...');

Output:

Starting fetch
Fetching...
undefined
{name: "emoutils",?…}

Eh, asynchronous function seems not "asynchronous", Async function body's first segment (part before first await) is synchronously executed, similar to:

new Promise(resolve => {
  console.log('Starting fetch');
  setTimeout(resolve.bind(null, 'data'), 100);
}).then(data => console.log(data));
console.log('Fetching...');

Similarly, very easy to change this thing to our pirated version:

asyncFunction(function* fetchJson(url) {
  try {
    console.log('Starting fetch');
    let request = yield fetch(url);
    let text = yield request.text();
    return JSON.parse(text);
  } catch(error) {
    console.error(error);
  }
}, 'https://unpkg.com/emoutils/package.json')
  .then(json => console.log(json));

// test
console.log('Fetching...');

In fact we did 3 things:

Wrap function body with Generator, change all await to yield
Remove space between async and function and camelCase naming
Move parameters behind Generator

If shield these 3 things through compilation transformation (even simple match replace can do):

function afunction(templateData) {
  const source = templateData;
  // ... some operations convert above string content to
  let params = ['url'];
  let transformed = `function* fetchJson(url) {
    try {
      console.log('Starting fetch');
      let request = yield fetch(url);
      let text = yield request.text();
      return JSON.parse(text);
    } catch(error) {
      console.error(error);
    }
  }`;

  return function(...args) {
    return asyncFunction(new Function(...params, `return ${transformed}`)(), ...args);
  };
}

Async function feature is completely replaced by pirated solution, syntax form can also become more similar:

afunction`(url) => {
  try {
    console.log('Starting fetch');
    let request = await fetch(url);
    let text = await request.text();
    return JSON.parse(text);
  } catch(error) {
    console.error(error);
  }
}`('https://unpkg.com/emoutils/package.json')
  .then(json => console.log(json));

P.S. Here applied ES2015 tagged templates feature, specifically see [Template Strings_ES6 Notes 3](/articles/模板字符串-es6 笔记 3/#articleHeader6)

Then, Is Async function syntax sugar?

Can consider yes. Because after having Generator feature, Async function is also about to emerge (from yield to await, essentially just further improved asynchronous programming experience, considered micro-improvement):

Internally, async functions work much like generators, but they are not translated to generator functions.

But language level feature support has more advantages than similar compilation transformation alternative solutions, reflected in performance, error tracking (clean call stack), seamless integration with other features (such as arrow functions, method definitions) etc.

Asynchronous Programming Experience

From programming experience perspective, Async function feature's brought improvement lies in:

Write asynchronous code in synchronous form, asynchronous, callback etc. concepts are diluted
try-catch can capture exceptions in asynchronous operations

Can let code blocks containing asynchronous operations still execute sequentially, this is undoubtedly the best asynchronous programming experience:

// callback reqXXX(parameters, success callback, failure callback)
reqLogin(password, reqOrderNo, notFound);
  reqOrderNo(uid, reqOrderDetail, notFound);
    reqOrderDetail(orderNo, render, boom);
      render(data);

// promise
promisifiedReqLogin(password)
  .then(({ uid }) => promisifiedReqOrderNo(uid), notFound)
  .then(({ orderNo }) => promisifiedReqOrderDetail(orderNo), notFound)
  .then(({ data }) => render(data))
  .catch(boom)

// async function
async function renderPage(password) {
  let uid, orderNo;
  try {
    uid = await promisifiedReqLogin(password);
    orderNo = await promisifiedReqOrderNo(uid);
  } catch(ex) {
    notFound(ex);
  }

  let data = await promisifiedReqOrderDetail(orderNo);
  return render(data);
}
renderPage().catch(boom);

data = await fetchData(), that's all. Callback concept no longer exists, reduced brain's burden following asynchronous operations pushing in and out of stack, after all

Code is written for people to read, incidentally can run on machines

(From [Write Good JavaScript](/articles/写好 javascript/))

5. Origin

So far, we have already used Generator and Promise features to implement pirated Async function, even didn't spend much effort (only 18 lines of code)

Now think back how we combined these two features? Or say, relying on which mechanisms of these two features, let piracy easily implement?

First, to implement Async function, most critical feature is Generator, through yield let sequential execution flow stop, only then have "wait" to speak of:

function* infSeq() {
  let i = 0;
  // Won't have infinite loop哟，yield let while true "stop" down
  while (true) {
    console.log(i);
    yield i++;
  }
}

// test
let iter = infSeq();
// Output 0, 1, 2...
iter.next();
iter.next();
iter.next();

Can "wait" now, then wait for whom? Directly wait for asynchronous operations? How to distinguish asynchronous operations?

Yes, time for Promise to appear:

// generator pretending async function
function* gen() {
  console.log('Do step 1');
  yield Promise.resolve('Until step1 completed');
  console.log('Do step 2');
}
let iter = gen();
let step1 = iter.next();
step1.value.then(iter.next.bind(iter));
console.log('generator pretending async function smokes a cigarette');

As long as next().value is Promise, then definitely is asynchronous operation, wait for it to complete then next(), this way implemented wait for one asynchronous operation to finish before continuing below things, i.e. Async function feature

From upper concept perspective, three parties relationship as follows:

Async function = Scheduler (Generator) + Asynchronous Task (Promise)

Among them, Generator this scheduler's role lies in:

Slicing (can't wait if can't split): Split function body sequential code blocks into several segments
Scheduling (how to execute after splitting): Control these segments' execution flow from outside, such as next(), throw() etc.

Promise as asynchronous task model, main characteristics as follows:

State discard: One-time Promise object, throw away after use (then() etc. all return new Promise)
Task combination: Can form task chain through similar resolve(promise) way, combine with all(), race() etc. to control its order
Error throw: Similar to bubbling exception handling mechanism, throw exceptions up along task chain, simplified asynchronous task exception capture

Generator doesn't directly schedule Promise (scheduling object is split segments), but it pays attention to each segment's execution result, if result is pending Promise, then wait until not pending, then control next segment execution

So, Promise is just supporting role, can be replaced with any asynchronous task model, its main role lies in informing Generator here has an asynchronous operation need to wait:

Scheduler: (poke a code segment on paper tape, stuff into computer, take out execution result) Eh, what's this?
Asynchronous Task: Hey, I'm an asynchronous task, not finished yet, I'll tell you when done
Scheduler: OK, I'll smoke a cigarette (avatar turns gray)
Asynchronous Task: Done done, result is xxx
Scheduler: (immediately online, pick up next code segment and xxx, both poke on paper tape, stuff into computer, take out execution result) Eh, this... damn, how come error appeared?