Common Issues in Chrome Extension Development

Foreword

I've been crafting a Chrome extension lately and encountered many issues. Now, I've finally moved into the actual development phase.

I. Goals and Requirements

Using interfaces as a demarcation line, frontend and backend development can proceed simultaneously. For the frontend, there are two stages:

1. Interfaces are not yet finished. Use mock data for the time being to debug pages and run logic.
2. Interfaces are finished. Remove the mock data and debug again using data provided by the backend services.

This implies that mock data must exist somewhere. Should it be written directly in the business code and commented out once the interfaces are done? Should it be extracted into an independent file, included in the page, and the script tag deleted later? Should a local service be set up specifically for mock data and replaced with the backend interface later? ... Putting it anywhere feels awkward because existing code must be modified once the interfaces are ready, and any modification brings the risk of errors.

Is there a way that requires no code changes?

Yes. Put the mock data in a Chrome extension. This way, the mock data and business code are completely separated.

II. Implementation Plan

Business code typically requests interfaces when the DOM is ready and displays the retrieved data on the page. Of course, some requests might be sent even before the DOM is ready.

Therefore, request interception logic and mock data should be injected into the page before any business code executes. The interception logic then distributes the corresponding mock data to the requests.

It seems necessary to intercept at least Ajax requests and JSONP GET requests. Interception itself isn't the problem; the key is filtering—allowing non-data requests to go through while intercepting data requests and returning mock data. If RESTful APIs are considered, mock data might become slightly more complex.

However, Hybrid Apps have some unique characteristics, for example:

• The client exposes certain functionalities as client interfaces for embedded pages to use.
• Data requests need to be forwarded by the client and encrypted by the client to provide security.

In this case, we don't need to worry about Ajax or JSONP. Since data requests are sent via client interfaces and data is "provided" by the client, we only need to simulate the client.

We don't even need to consider request interception; we can just directly impersonate the client to provide data request interfaces (of course, we can also provide other functional interfaces; with a full set, debugging would hardly depend on the client at all).

P.S. Chrome extensions provide permissions like "webRequest", "webRequestBlocking" to handle page requests. For more information, please check the official documentation.

After deciding on the plan, we need to consider the details:

• iOS typically provides native interfaces through the third-party library WebViewJavascriptBridge.
• The simplest way for Android to provide interfaces is to expose a global object to JS.

The "native ready" notification for WebViewJavascriptBridge works by triggering a custom event WebViewJavascriptBridgeReady on the document, which the page JS listens to for "connection." Android is much more straightforward; the injected global object is accessible at any time. By the time the page JS executes, it is already "native ready," so no "connection" is needed.

Thus, the method to impersonate the client is: for Android, attach the custom global object before any page JS executes; for iOS, manually trigger WebViewJavascriptBridgeReady at any time. Therefore, Android requires synchronous injection of mock interfaces, whereas iOS does not. Mock data itself does not need to be injected synchronously (mock data might be large; asynchronous injection keeps the page faster, and requests can be queued before the mock data arrives).

III. Issues and Solutions

1. content_scripts don't work on local files

In the extension management page, some extensions have a checkbox labeled "Allow access to file URLs." This option is hidden by default. It only appears if the following is present in the manifest.json configuration file:

"content_scripts": [{
  "matches": ["<all_urls>"]
}]

Once checked, content_scripts can process local files.

2. How to inject scripts synchronously?

A content script can manipulate the page DOM but cannot directly modify JS variables because the two have different JS execution environments. To modify page JS, you must inject a script tag. This involves two steps:

1. Retrieve the script to be injected into the page.
2. Create a script tag and insert it into the page.

Asynchronous script injection is very easy:

var loadScript = function(filename) {
    var s = document.createElement('script');
    // Get the file:/// path of mock.js after the extension is installed
    //! web_accessible_resources must be declared in manifest.json
    s.src = chrome.extension.getURL(filename);
    // If head is not ready, use html
    var doc = document.head || document.documentElement;
    return doc.appendChild(s);
}
// test
loadScript('mock.js');

A script tag will be inserted into the page with a src attribute like file:///xxx. The script file will load asynchronously and execute upon completion.

In reality, this script executes after DOM ready, and requests made before then will slip away because the mock client is still loading. We must ensure the mock logic executes before all page JS to prevent requests from slipping away under the Android UA.

This requires a synchronous script injection plan (executing immediately upon injection rather than loading asynchronously). We can synchronously read the script and then insert an inline script, as follows:

// 1. Synchronously retrieve script content
var readFileSync = function(filename, callback) {
    // read script sync
    var xhr = new XMLHttpRequest();
    var scriptUrl = chrome.extension.getURL(filename);
    //!!! disable async
    xhr.open("GET", scriptUrl, false);
    xhr.onreadystatechange = function() {
        if (xhr.readyState === 4 && xhr.status === 200) {
            callback(xhr.responseText);
        }
    };
    xhr.send(null);
};
// 2. Insert inline script
var writeScriptSync = function(code) {
    var s = document.createElement('script');
    s.textContent = code;
    var doc = document.head || document.documentElement;
    return doc.appendChild(s);
};
// test
readFileSync('mock.js', writeScriptSync);

The key lies in xhr.open("GET", scriptUrl, false); and s.textContent = code;. The former uses a synchronous Ajax call to read the file content, and the latter writes the code string into the script tag.

3. Toolbar Icon and Extension Management Page Icon

A proper extension needs an icon. You need to set the browser_action.default_icon and icons fields in the manifest.json file respectively, as shown below:

// Toolbar icon
"browser_action": {
  "default_icon": "icon/icon.png",
},
// Extension management page icon
"icons": {
  "128": "icon/128.png",
  "16": "icon/16.png",
  "48": "icon/48.png"
}

4. Data Read/Write

There are many optional solutions:

• window.localStorage: Synchronous read/write, subject to CORS and 5MB limits.
• chrome.storage.local/sync: Asynchronous read/write, subject to a 5MB limit (can be removed with permission declaration).
• IndexedDB: Asynchronous read/write, a NoSQL database.
• WebSQL: Asynchronous read/write, an SQLite database, non-standard.

Non-standard WebSQL is not considered. window.localStorage has no major advantage. chrome.storage.local/sync is slightly more powerful, but its read/write efficiency is lower than IndexedDB. Therefore, it is recommended to choose IndexedDB.

Additionally, each has its advantages:

• window.localStorage is synchronous, which is useful in certain scenarios requiring synchronicity.
• chrome.storage.local is the simplest way, meeting the needs for small amounts of data storage.
• chrome.storage.sync can automatically sync across multiple devices, which is very powerful.

In practice, I found that chrome.storage.sync has set failure issues (retrieving undefined with get), so I've temporarily switched to local for state data. Declaring the unlimitedStorage permission can remove the 5MB limit, but it brings new restrictions:

Note: This permission applies only to Web SQL Database and application cache (see issue 58985). Also, it doesn't currently work with wildcard subdomains such as http://*.example.com.

P.S. The standardization process for WebSQL has stalled because all browsers were based on SQLite implementations, and the process became restricted by SQLite, leading to a deadlock. For more information, please check Web SQL Database.

5. eval Permission

By default, extensions are not allowed to use eval to execute content from external files. For instance, when an extension is installed, reading configuration data and writing it to storage using eval will throw an error. If you must use it, you need to modify the CSP (Content Security Policy) in manifest.json:

"content_security_policy": "script-src 'unsafe-eval'; object-src 'self'"

The default script-src 'self'; object-src 'self' does not allow unsafe eval.

IV. IndexedDB

The native API is quite cumbersome to use and requires understanding various strange concepts: objectStore, keyPath, transaction, etc. It is recommended to use the third-party library Dexie.js as a DBHelper.

Its API design is very elegant. Examples of CRUD (Create, Read, Update, Delete) are as follows:

// Create DB and tables
var db = new Dexie('db');
db.version(1).stores({
    // ++ indicates an auto-incrementing field, & indicates a unique constraint
    tb: '++id, &key, value, desc, other'
});
// Create
db.tb.bulkAdd([{
    key: 1,
    value: '1'
}, {
    key: 2.5,
    value: '2.5'
}, {
    key: 2,
    value: '2'
}]);
// Delete
db.tb
    .where('key')
    .equals(2.5)
    .toArray()
    .then(function(res) {
        console.log(res);
        // Delete
        db.tb.delete(res[0].id);
    }, console.log.bind(console));
// Update
db.tb.put({
    key: 'getPoiInfo',
    value: 'value'
});
// Read
db.tb
    .where('id')
    .inAnyRange([[0, 100]])
    .each(function(item) {
        console.log(item);
    });
// Read + Update
db.tb
    .where('key')
    .equals(1)
    .modify({value: 'value1'});
// Custom full-table search
db.tb
    .filter(item => /get.*/i.test(item.key))
    .each(function(item) {
        console.log(item.value);
    });

V. Script Communication Methods

There are multiple types of scripts in extension development, and communication between them is necessary, as follows:

• Communication between content scripts and "scripts belonging entirely to the extension."
• Communication between injected scripts and content scripts.
• Communication among "scripts belonging entirely to the extension."
• Communication between the extension and external servers.

Our focus is on the communication between content scripts and "scripts belonging entirely to the extension" because content scripts need to request mock data from the background page. This can be achieved through runtime message:

// Establish a communication connection
var port = chrome.runtime.connect({name: "app"});
// Send from content script
port.postMessage({
    action: 'KEYQUERY',
    key: key
});
// Receive in background page background.js
chrome.runtime.onConnect.addListener(function(port) {
    port.onMessage.addListener(function(msg) {
        if (port.name != "app") return;
        switch (msg.action) {
            case 'KEYQUERY':
                db.tb
                    .filter(item => item.key === msg.key)
                    .toArray()
                    .then(function(arr){
                        port.postMessage({
                            _action: msg.action,
                            data: arr
                        });
                    }, function() {
                        port.postMessage({
                            _action: msg.action,
                            data: []
                        });
                    });
        }
    }
}

In the example above, a connection is established first, followed by message exchange. You can also send single messages without establishing a connection:

// Send
chrome.runtime.sendMessage({greeting: "hello"}, function(response) {
    console.log(response.farewell);
});
// Receive
chrome.runtime.onMessage.addListener(
    function(request, sender, sendResponse) {
        console.log(sender.tab ?
                "from a content script:" + sender.tab.url :
                "from the extension");
        if (request.greeting == "hello")
        sendResponse({farewell: "goodbye"});
    }
);

However, note that single messages cannot respond asynchronously because there is no persistent connection. For example:

// Receive
chrome.runtime.onMessage.addListener(
    function(request, sender, sendResponse) {
        console.log(sender.tab ?
                "from a content script:" + sender.tab.url :
                "from the extension");
        if (request.greeting == "hello")
        // Asynchronous response
        setTimeout(() => {
            sendResponse({farewell: "goodbye"});
        }, 1000);
        //!!! This will immediately trigger the sender's callback, returning undefined
        // The sendResponse after 1 second is lost
    }
);

In this case, the asynchronous sendResponse will be lost. Therefore, it is recommended to communicate by establishing a persistent connection.

For more information on script communication methods, please check Chrome Extension Development Part 2: Script Running Mechanisms and Communication Methods in Chrome Extensions.

References

• My injected <script> runs after the target-page's javascript, despite using run_at: document_start?

• Building a Chrome Extension - Inject code in a page using a Content script