Track web browser usage in Android using Accessibility Service

Accessibility Service, although meant to assist users with disabilities, is a powerful tool to track user behaviour. It gives us the ability to track user gestures, monitor app usage and also read text from various apps among other things.

We can track the urls visited by a user by implementing an Accessibility Service listener in the following way. It starts by defining our service in AndroidManifest.xml file in the following way.

<service android:name=".LogUrlService"
    android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
    <intent-filter>
        <action android:name="android.accessibilityservice.AccessibilityService" />
    </intent-filter>
    <meta-data android:name="android.accessibilityservice" android:resource="@xml/accessibilityservice" />

</service>

Details on what kind of events we want to listen will go in an xml file named accessibilityservice .Here are the contents of that file. The event we are interested in are typeWindowsChanged ,typeWindowStateChanged & typeWindowContentChanged. Another important property is canRetrieveWindowContent which is set to true.

<?xml version="1.0" encoding="utf-8"?>
<accessibility-service xmlns:android="http://schemas.android.com/apk/res/android"
    android:accessibilityEventTypes = "typeWindowsChanged|typeWindowStateChanged|typeWindowContentChanged"
    android:accessibilityFeedbackType="feedbackVisual"
    android:notificationTimeout="300"
    android:accessibilityFlags="flagDefault|flagIncludeNotImportantViews|flagRequestTouchExplorationMode|flagRequestEnhancedWebAccessibility|flagReportViewIds|flagRetrieveInteractiveWindows"
    android:canRetrieveWindowContent="true"

    >

</accessibility-service>

Permission to access this service needs to be explicitly granted to your app by the user, by navigating to Settings->Apps&Notifications>Advanced>Special App usage>Accessibility or we can do this programatically by calling the accessibility settings intent

Intent intent = new Intent(Settings.ACTION_ACCESSIBILITY_SETTINGS);
startActivity(intent);

Once permission has been granted, our service starts receiving callbacks whenever the accessibility events are trigerred. Since this event will be triggered by all apps, we need to filter it to ensure that we are only processing events related to a web browser. We need to know beforehand the package name of the browser we are going to track. Every time an event is triggered, we are presented with an AccessibilityNodeInfo object, which includes the window content as well as a tree of the child AccessibilityNodeInfo of this parent object. We look for a particular node , which represents the addressbar of that browser. This is unique for every browser. This is a one time activity which we get by traversing through the nodes and figuring out which node contains the data and get the associated node id name.

private void getChild(AccessibilityNodeInfo info)
{
    int i=info.getChildCount();
    for(int p=0;p<i;p++)
    {
        AccessibilityNodeInfo n=info.getChild(p);
        if(n!=null) {
            String strres = n.getViewIdResourceName();
            if (n.getText() != null) {
                String txt = n.getText().toString();
                Log.d("Track", strres + "  :  " + txt);
            }
            getChild(n);
        }
    }
}

Once this is done , we can build a list with the package names and the unique node id name. Here is a list with a few popular browsers on android

private static List<SupportedBrowserConfig> getSupportedBrowsers() {
    List<SupportedBrowserConfig> browsers = new ArrayList<>();
   
 browsers.add( new SupportedBrowserConfig("com.android.chrome", "com.android.chrome:id/url_bar"));
 
   browsers.add( new SupportedBrowserConfig("org.mozilla.firefox", "org.mozilla.firefox:id/mozac_browser_toolbar_url_view"));
 
   browsers.add( new SupportedBrowserConfig("com.opera.browser", "com.opera.browser:id/url_field"));
 
   browsers.add( new SupportedBrowserConfig("com.opera.mini.native", "com.opera.mini.native:id/url_field"));
 
   browsers.add( new SupportedBrowserConfig("com.duckduckgo.mobile.android", "com.duckduckgo.mobile.android:id/omnibarTextInput"));
 
   browsers.add( new SupportedBrowserConfig("com.microsoft.emmx", "com.microsoft.emmx:id/url_bar"));


    return browsers;
}

Now all that is left is to query for the content of that particular node whenever the browser window becomes active or changes.

case AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED:


case AccessibilityEvent.TYPE_WINDOWS_CHANGED:
case AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED: {
    AccessibilityNodeInfo parentNodeInfo = event.getSource();
    if (parentNodeInfo == null) {
        return;
    }

    String packageName = event.getPackageName().toString();
    SupportedBrowserConfig browserConfig = null;
    for (SupportedBrowserConfig supportedConfig: getSupportedBrowsers()) {
        if (supportedConfig.packageName.equals(packageName)) {
            browserConfig = supportedConfig;
        }
    }
    if (browserConfig == null) {
        return;
    }

    String capturedUrl = captureUrl(parentNodeInfo, browserConfig);
    parentNodeInfo.recycle();

    if (capturedUrl == null) {
        return;
    }

Now we can continuously monitor in the background all the urls visited by a user even if the browser is operated in incognito mode.

You can find the source files of this project here