Part 1 and Part 2 of this series examined how the Android framework code, as modified by Samsung, forced a Mainland Chinese DNS service upon Samsung phone users with Hong Kong firmware and those connected to Hong Kong mobile network, as well as how periodic DNS queries were made for qq.com to the DNS servers (without actually connecting to the qq.com site).
However, this could not explain all those DNS queries to other popular Chinese websites like baidu.com (whereas such queries were also observed to cease when a Samsung phone was tricked into stopping the periodic DNS queries by changing the WiFi SSID, meaning that these queries are likely to be associated with Samsung firmware). After some examination, the culprit for the remaining unwanted queries is found – the Network Stack module as modified by Samsung, under /system/priv-app/InProcessNetworkStack/InProcessNetworkStack.jar.
The InProcessNetworkStack.jar module of Samsung firmware has been decompiled and uploaded to the github repository SM-9750-TGY-Oct20-Network.
In short, the class NetworkMonitor under the module, which is modified from Android project source here, is responible for the behaviour, including sending DNS queries and actually connecting to and / or sending plain HTTP requests to one or more of the following URLs to perform connectivity probes, when WiFi is connected to a network selected by the user, and the code determines that it is in a “China network”:
http://www.qq.comhttp://www.baidu.comhttp://m.taobao.comhttp://m.hao123.comhttp://connectivity.samsung.com.cn/generate_204
It is also found that besides WiFi users in Hong Kong, Macau and Mainland China, worldwide users of Samsung WiFi-only devices (e.g. tablets) might be regarded as in a “China network” if there are sufficient access points imported from China or Hong Kong nearby, as determined by a voting algorithm specific to Samsung devices. Connection to those access points is not necessary for this to occur.
In cases when the device does not consider itself to be in a “China network”, it would stick to the normal behaviour of doing the test with the url as configured in the Android Global Settings captive_portal_http_url and captive_portal_https_url, which would usually be Google owned sites [1].
Update: the new version of firmware for Hong Kong, released on 23 October (for Note 20 Ultra, firmware dated 20 October) has addressed the issue of treating “hk” the same way as “cn”, addressing local concerns. However, the issue described below will still affect phones that are for Mainland China, connected to “cn” network, or near “cn” access points, after the firmware update.
The following is again a detailed walkthrough of how the probes above are triggerred and sent. A method to avoid these requests in affected Samsung devices is provided at the end of the article.
Triggers from WifiConnectivityMonitor to NetworkMonitor
Before going into how Samsung modified NetworkMonitor to launch probes on the Mainland China websites instead of the default Google site, we will first have a tour of the WifiConnectivityMonitor (hereafter shortened as WCM) code which initiated the probes. The version used to illustrate in this Part is from the updated Note 10+ firmware (Hong Kong version, dated 10 October 2020), uploaded to Github at WifiConnectivityMonitor.java [2]
Under the WCM, the call to NetworkMonitor is triggered by methods reportNetworkConnectivityToNM(). They report to the ConnectivityManager, a standard Android system service, that a connectivity check is needed as there may be changes in the circumstances of the WiFi connection.
These methods are from line 9149–
public boolean reportNetworkConnectivityToNM(int step, int trigger) {
return reportNetworkConnectivityToNM(false, step, trigger);
}
public boolean reportNetworkConnectivityToNM(boolean force, int step, int trigger) {
if ((!force && trigger != QC_TRIGGER_AT_EVENT_ROAM_COMPLETE && ((!this.mIsScreenOn && mInitialResultSentToSystemUi) || isMobileHotspot())) || this.mIsInRoamSession || this.mIsInDhcpSession) {
return false;
}
if (this.mNetwork != null) {
if (this.mWCMQCResult != null) {
Log.d(TAG, "QC is already queried to NM");
return true;
}
Log.d(TAG, "QC is queried to NM. Waiting for result");
removeMessages(QC_RESULT_NOT_RECEIVED);
this.mWCMQCResult = obtainMessage();
getCm().reportNetworkConnectivityForResult(this.mNetwork, this.mWCMQCResult);
QcFailHistory qcFailHistory = this.mInvalidationFailHistory;
qcFailHistory.qcStep = step;
qcFailHistory.qcTrigger = trigger;
qcFailHistory.qcStepTemp = -1;
sendMessageDelayed(QC_RESULT_NOT_RECEIVED, 20000);
if (isValidState()) {
setLoggingForTCPStat(TCP_STAT_LOGGING_FIRST);
}
}
return true;
}
The main method first checks the pre-conditions for triggering the network connectivity test, i.e. force checking (force) is true, OR the call is triggered from a WiFi Roam Complete event (QC_TRIGGER_AT_EVENT_ROAM_COMPLETE), OR screen is on, OR the initial detection result after connection was not yet available (mInitialResultSentToSystemUi is false). But in any case it would not call the Network Monitor when the WiF connection is used as a mobile hotspot, is doing WiFi Roaming, or is querying DHCP.
After some further checking, it calls the reportNetworkConnectivityForResult() method of the Android ConnectivityManager. The reportNetworkConnectivityForResult() method, which is a Samsung made variant of the standard reportNetworkConnectivity() method, is for prompting the system to re-evaluate the network’s connectivity (ultimately done by the NetworkMonitor) and other further actions as a result.
Tracing backwards, this reportNetworkConnectivityToNM() method is mainly invoked under two processes, which in turn are triggered by a lot of scenarios. These are briefly outlined below.
Internet Check
The “Internet Check” is invoked by calling requestInternetCheck() (line 4143). It will send the REPORT_NETWORK_CONNECTIVITY message to the WifiConectivityMonitor, after some usual checking.
Line 4137 onwards:
private void requestInternetCheck(int trigger) {
requestInternetCheck(this.mInvalidationFailHistory.qcStepTemp, trigger);
}
private void requestInternetCheck(int step, int trigger) {
WifiEleStateTracker wifiEleStateTracker = this.mWifiEleStateTracker;
if (wifiEleStateTracker != null && wifiEleStateTracker.checkEleValidBlockState()) {
Log.d(TAG, "REPORT_NETWORK_CONNECTIVITY ignored by ele block");
} else if (this.mIsInRoamSession || this.mIsInDhcpSession) {
Log.d(TAG, "REPORT_NETWORK_CONNECTIVITY ignored In Roam Session");
} else if (syncGetCurrentWifiInfo().getRssi() >= -55 || !this.mIsRoamingNetwork) {
sendMessage(REPORT_NETWORK_CONNECTIVITY, step, trigger);
} else {
Log.d(TAG, "REPORT_NETWORK_CONNECTIVITY ignored by possible roaming");
}
}
When this message is received under the InvalidState (line 2905), ValidNonSwitchableState (line 3207) or ValidSwitchableState (line 3367) of the WCM, reportNetworkConnectivityToNM() will be called without setting the force parameter. A typical handling is shown below. Given the state hierarchy (lines 1061-1072), these states cover all the ConnectedState, except CaptivePortalState and ValidNoCheckState (i.e. mCurrentMode is 0).
case WifiConnectivityMonitor.REPORT_NETWORK_CONNECTIVITY:
WifiConnectivityMonitor.this.reportNetworkConnectivityToNM(msg.arg1, msg.arg2);
break;
Forced Validation Check
“Forced validation check” is invoked when enableValidationCheck() (line 11120) is called. It will set mValidationCheckMode and call reportNetworkConnectivityToNM() once, with force parameter set to true (i.e. always check even if screen is off, except in transient or hotspot modes). Then, if mValidationBlock has been set to true, it will send the VALIDATION_CHECK_FORCE message after a delay.
private void enableValidationCheck() {
if (DBG) {
Log.d(TAG, "ValidationCheckMode : " + mValidationCheckMode + ", ValidationCheckCount : " + mValidationCheckCount + ", ValidationBlock : " + mValidationBlock);
}
if (!mValidationCheckMode || mValidationCheckCount <= 0) {
if (DBG) {
Log.d(TAG, "Validation Check enabled.");
}
mValidationResultCount = 0;
mValidationCheckCount = 0;
mValidationCheckMode = true;
mValidationCheckTime = VALIDATION_CHECK_COUNT; // 32
Message message = this.mWCMQCResult;
if (message != null) {
message.recycle();
this.mWCMQCResult = null;
}
removeMessages(QC_RESULT_NOT_RECEIVED);
boolean queried = reportNetworkConnectivityToNM(true, 5, 20);
if (!mValidationBlock) {
return;
}
if (queried) {
mValidationCheckCount = 1;
if (DBG) {
Log.d(TAG, "mValidationCheckCount : " + mValidationCheckCount);
}
sendMessageDelayed(VALIDATION_CHECK_FORCE, (long) (mValidationCheckTime * 1000));
return;
}
if (DBG) {
Log.d(TAG, "Starting to check VALIDATION_CHECK_FORCE is delayed.");
}
mValidationCheckTime = VALIDATION_CHECK_COUNT * 2;
sendMessageDelayed(VALIDATION_CHECK_FORCE, 10000);
} else if (DBG) {
Log.d(TAG, "Validation Check was already enabled.");
}
}
When this message is received under InvalidState of the WCM, it would continue to call reportNetworkConnectivityToNM and then resend the VALIDATION_CHECK_FORCE message to trigger itself again after a delay [3]. In order words, this validation check would form a loop under ValidationBlock mode. The loop is shown below (line 2922-)
case WifiConnectivityMonitor.VALIDATION_CHECK_FORCE:
if (WifiConnectivityMonitor.DBG) {
Log.d(WifiConnectivityMonitor.TAG, "VALIDATION_CHECK_FORCE");
}
WifiConnectivityMonitor.this.mValidationCheckEnabledTime = SystemClock.elapsedRealtime();
WifiConnectivityMonitor.this.mValidationCheckTime /= 2; // ??
WifiConnectivityMonitor.this.mValidationCheckCount++;
if (WifiConnectivityMonitor.DBG) {
Log.d(WifiConnectivityMonitor.TAG, "mValidationCheckCount : " + WifiConnectivityMonitor.this.mValidationCheckCount);
}
if (WifiConnectivityMonitor.this.mWCMQCResult != null) {
WifiConnectivityMonitor.this.mWCMQCResult.recycle();
WifiConnectivityMonitor.this.mWCMQCResult = null;
}
WifiConnectivityMonitor.this.removeMessages(WifiConnectivityMonitor.QC_RESULT_NOT_RECEIVED);
boolean queried = WifiConnectivityMonitor.this.reportNetworkConnectivityToNM(true, 5, 20);
if (WifiConnectivityMonitor.this.mValidationCheckCount > WifiConnectivityMonitor.VALIDATION_CHECK_MAX_COUNT /* 4 */) {
WifiConnectivityMonitor.this.mValidationCheckMode = false;
WifiConnectivityMonitor.this.setValidationBlock(false);
if (WifiConnectivityMonitor.DBG) {
Log.d(WifiConnectivityMonitor.TAG, "mValidationCheckCount expired");
}
if (queried) {
WifiConnectivityMonitor.this.mValidationResultCount = 0;
WifiConnectivityMonitor.this.mValidationCheckCount = 0;
} else {
WifiConnectivityMonitor.this.sendMessageDelayed(WifiConnectivityMonitor.VALIDATION_CHECK_FORCE, 10000);
}
return true;
}
WifiConnectivityMonitor.this.sendMessageDelayed(WifiConnectivityMonitor.VALIDATION_CHECK_FORCE, (long) (WifiConnectivityMonitor.this.mValidationCheckTime * 1000));
return true;
The loop will end when the CONNECTIVITY_VALIDATION_RESULT message is received (line 2908), i.e. checking result is available, by removing the VALIDATION_CHECK_FORCE message in the queue.
Tracing further back, the above two processes are invoked under numerous circumstances. The validation check mainly comes from a loop to keep monitoring the signal strength, when the connection is not yet valid. The Internet check comes from a wider range of sources, including from inside the loop discussed in Part 2 which keeps making minutely DNS queries of qq.com. While I would not go into further details on these circumstances, it would suffice to note in general that various connection status or signal quality changes could trigger the calls, and generally a lower mCurrentMode value (such as when the SSID is set to match the pattern of ignored network, forcing mCurrentMode to 0) would reduce the frequency of such calls.
Other signals from WCM to NetworkMonitor
It appears that the WCM is built by Samsung separately from the NetworkMonitor and ClientModeImpl of standard Android, as there is some overlap of functions among them. Perhaps as a result of this, the WCM, in addition to the conventional method of calling NetworkMonitor through ConnectivityManager above, also communicates with the NetworkMonitor through the Android Global Settings (i.e. Settings.Global which, as its name implies, are usually for longer-term system settings). The settings concerned include the following:
Captive portal mode
The captive_portal_mode is a flag in standard Android settings for controlling how the Network Monitor dealt with captive portals. Normally, setting it to 0 using adb would disable the detection (See this stackexchange answer). However, in Samsung phones the value of this setting would be overwritten by the WCM at various stages of establishing a WiFi connection to control the behaviour of Network Monitor, making it unavailable for setting by users.
The setting is modified in the WCM at lines 9746–
public void setCaptivePortalMode(int enabled) {
Settings.Global.putInt(this.mContext.getContentResolver(), WCMCallbacks.CAPTIVE_PORTAL_MODE, enabled);
}
WCM sets the value to 1 (enabled) in most of the circumstances, except a callback from checkIsCaptivePortalException() method of ClientModeImpl (line 9489-), which would be called upon entry to the connected state. This may set it to 0 (disable) depending on external factors such as SSID (including the case when isIgnorableNetwork(), discussed in Part 2, returns true).
Country code from scan results
A more interesting setting, which mainly affects WiFi-only Samsung products as to be discussed below, is wifi_wcm_country_code_from_scan_result. It holds the outcome of a voting mechanism for determining the country code of the device using purely WiFi scanning results (since the country code from mobile network is not available for these devices).
Before going through Samsung’s algorithm, here is a brief background of the relevant WiFi protocol. Under the 802.11 standard, WiFi signals would contain management frames which consist of a number of “Information Elements” (IEs). These IEs have a common format, starting with the Element ID which encodes what type of information is stored with this IE, followed by the field length and actual data.
In particular, an ID value of 7 denotes the “country information element“. This IE is used to store country code data (with the country code in the first 2 bytes of data) among other information. Presumably, WiFi routers are required to know the country code in which they are operating, such that they can ensure that the channels used and the signal strength is in compliance with the local laws.
While this information is broadcast by WiFi access points, it is usually not accessible to the users. However, system software at privileged levels, like the WCM, has access to the IE contents in scan results, including the country code, if they are present.
The first part of the checkCountryCodeFromScanResults() method of the WCM (line 8738-), shown below, gathers all country code information of WiFi access points that are scanned, and counts the occurrence of each country code – with a twist: it specifically looks for HK and MO (standing for Hong Kong and Macau), and if encountered, replaces it with CN before counting.
Note that this is a separate piece of code from what was discussed in Part 1 (which still exists in the latest firmware as at October 2020), both making a similar assumption. The different places and forms that the assumption exists throughout the Samsung framework code (spoiler: the third one will be coming) somehow suggests that this might not be an unintentional mistake.
public void checkCountryCodeFromScanResults() {
int scanCount = 0;
int bssidWithCountryInfo = 0;
HashMap<String, Integer> countryCodeMap = new HashMap<>();
StringBuilder sb = new StringBuilder();
synchronized (mScanResultsLock) {
for (ScanResult scanResult : mScanResults) {
scanCount++;
ScanResult.InformationElement[] informationElementArr = scanResult.informationElements;
int length = informationElementArr.length;
int i = 0;
while (true) {
if (i >= length) {
break;
}
ScanResult.InformationElement ie = informationElementArr[i];
if (ie.id == 7) {
bssidWithCountryInfo++;
if (ie.bytes.length >= 2) {
String country = new String(ie.bytes, 0, 2, StandardCharsets.UTF_8).toUpperCase();
if ("HK".equals(country) || "MO".equals(country)) {
country = "CN";
}
if (countryCodeMap.containsKey(country)) {
countryCodeMap.put(country, Integer.valueOf(countryCodeMap.get(country).intValue() + 1));
} else {
countryCodeMap.put(country, 1);
}
}
} else {
i++;
}
}
}
}
/**
* remaining code below
*/
}
After gathering the statistics of nearby WiFi networks, with HK/MO replaced by and counted as CN, the code selects the “winner”, i.e. the country code with the highest count. It specifically ensures that if this ends in a tie, with CN and another country code having the same count, CN wins.
Then, if winner has at least 5 counts, the record kept by the variable mCountryCodeFromScanResult and the wifi_wcm_country_code_from_scan_result global setting would be updated – again with a special rule: if the previous winner is CN but the new winner is anyone else, it would not recognize the new winner unless the previous CN record was more than 24 hours old. (Or the phone has been switched off since then, thus resetting the time).
The remaining code from line 8773–
if (bssidWithCountryInfo != 0) {
int winnerCount = 0;
if (countryCodeMap.containsKey("CN")) {
countryCodeMap.get("CN").intValue();
}
String pollWinner = "";
String stat = "";
for (String c : countryCodeMap.keySet()) {
stat = (stat + c + ": " + countryCodeMap.get(c)) + " ";
if (countryCodeMap.get(c).intValue() > winnerCount || (countryCodeMap.get(c).intValue() >= winnerCount && "CN".equals(c))) {
pollWinner = c;
winnerCount = countryCodeMap.get(c).intValue();
}
}
StringBuilder sb = new StringBuilder();
...
if (winnerCount >= 5) {
if ("CN".equals(pollWinner)) {
mLastChinaConfirmedTime = System.currentTimeMillis();
}
if ("CN".equals(mCountryCodeFromScanResult) && !"CN".equals(pollWinner)) {
long remainingTime = 86400000 - (System.currentTimeMillis() - mLastChinaConfirmedTime);
if (remainingTime < 0) { sb.append(" | CISO Updated [24h expired] - CN -> " + pollWinner);
mCountryCodeFromScanResult = pollWinner;
Settings.Global.putString(this.mContext.getContentResolver(), "wifi_wcm_country_code_from_scan_result", mCountryCodeFromScanResult);
} else {
sb.append(" | CISO changed but not updated - CN -X-> " + pollWinner + " , maintain CN for next " + (remainingTime / 1000) + " seconds");
}
} else if (!pollWinner.equals(mCountryCodeFromScanResult)) {
sb.append(" | Updated - " + mCountryCodeFromScanResult + "->" + pollWinner);
mCountryCodeFromScanResult = pollWinner;
Settings.Global.putString(mContext.getContentResolver(), "wifi_wcm_country_code_from_scan_result", mCountryCodeFromScanResult);
}
}
...
}
Roam complete
Yet another signal the WCM sends to the NetworkMonitor via global settings is the Roam Complete event. The wifi_wcm_event_roam_complete setting was set in the setRoamEventToNM() method. Under the WCM, it will be set to 1 after the WiFi roam session is completed. After the NetworkMonitor “comsumed” the signal, it would be reset to 0 by the NetworkMonitor. It is also reset by WCM upon WiFi disconnection.
private void setRoamEventToNM(int enable) {
Settings.Global.putInt(this.mContentResolver, "wifi_wcm_event_roam_complete", enable);
}
Enter the Network Monitor
The reportNetworkConnectivityForResult() call from the WCM above will eventually reach the ConnectivityService, and is handled by the method handleReportNetworkConnectivityForResult() under the class (this is decompiled from the services.jar located at /system/framework. The code is similar to the standard Android method handleReportNetworkConnectivity()). This method will call the forceReevaluation() method of NetworkMonitor associated with the WiFi network. (Note that the standard Android method handleReportNetworkConnectivity(), which can be invoked by user Apps, will also end up at forceReevaluation(). So the WCM is not the only source to invoke the connectivity probe of NetworkMonitor to be discussed below).
private void handleReportNetworkConnectivityForResult(Network network, int uid) {
NetworkAgentInfo nai;
if (network == null) {
nai = getDefaultNetwork();
} else {
nai = getNetworkAgentInfoForNetwork(network);
}
if (nai != null && nai.networkInfo != null && nai.networkInfo.getType() == 1 && this.mUserWantAsIs) {
log("User want the CaptivePortal network as is. Ignore reportNetworkConnectivity for Wi-Fi network");
} else if (nai != null && nai.networkInfo.getState() != NetworkInfo.State.DISCONNECTING && nai.networkInfo.getState() != NetworkInfo.State.DISCONNECTED) {
int netid = nai.network.netId;
log("reportNetworkConnectivityForResult(" + netid + ") by " + uid);
if (nai.everConnected && !isNetworkWithLinkPropertiesBlocked(getLinkProperties(nai), uid, false)) {
nai.networkMonitor().forceReevaluation(uid);
}
}
}
The NetworkMonitor States
Like the WCM, the NetworkMonitor is also a StateMachine. It is present in standard Android and is responsible for, among others, probing internet connectivity for the detection of captive portal, and launching a browser for users to dealt with a captive portal. Its states include EvaluatingState, ProbingState, WaitingForNextProbeState, CaptivePortalState, etc. The state hierarchy is at line 263 –
addState(this.mDefaultState);
addState(this.mMaybeNotifyState, this.mDefaultState);
addState(this.mEvaluatingState, this.mMaybeNotifyState);
addState(this.mProbingState, this.mEvaluatingState);
addState(this.mWaitingForNextProbeState, this.mEvaluatingState);
addState(this.mCaptivePortalState, this.mMaybeNotifyState);
addState(this.mEvaluatingPrivateDnsState, this.mDefaultState);
addState(this.mValidatedState, this.mDefaultState);
setInitialState(this.mDefaultState);
From the initial DefaultState, the NetworkMonitor will enter EvaluatingState upon connection (line 427), as well as when a CMD_FORCE_REEVALUATION message is received at line 484.
Of relevance to us is the CMD_FORCE_REEVALUATION message, which would be sent by the call to the forceReevaluation() method (at line 295), invoked from the ConnectivityService above:
public void forceReevaluation(int responsibleUid) {
sendMessage(CMD_FORCE_REEVALUATION /*8*/, responsibleUid, 0);
}
Upon entry to the EvaluatingState, it will send the CMD_REEVALUATE message to itself (line 624).
So far, the code and the flow in the Network Monitor is largely the same as standard Android. The part handling the CMD_REEVALUATE message as taking from Android source, with helpful comments, at line 1245 of NetworkMonitor.java is as follows:
case CMD_REEVALUATE:
if (message.arg1 != mReevaluateToken || mUserDoesNotWant) {
return HANDLED;
}
// Don't bother validating networks that don't satisfy the default request.
// This includes:
// - VPNs which can be considered explicitly desired by the user and the
// user's desire trumps whether the network validates.
// - Networks that don't provide Internet access. It's unclear how to
// validate such networks.
// - Untrusted networks. It's unsafe to prompt the user to sign-in to
// such networks and the user didn't express interest in connecting to
// such networks (an app did) so the user may be unhappily surprised when
// asked to sign-in to a network they didn't want to connect to in the
// first place. Validation could be done to adjust the network scores
// however these networks are app-requested and may not be intended for
// general usage, in which case general validation may not be an accurate
// measure of the network's quality. Only the app knows how to evaluate
// the network so don't bother validating here. Furthermore sending HTTP
// packets over the network may be undesirable, for example an extremely
// expensive metered network, or unwanted leaking of the User Agent string.
//
// On networks that need to support private DNS in strict mode (e.g., VPNs, but
// not networks that don't provide Internet access), we still need to perform
// private DNS server resolution.
if (!isValidationRequired()) {
if (isPrivateDnsValidationRequired()) {
validationLog("Network would not satisfy default request, "
+ "resolving private DNS");
transitionTo(mEvaluatingPrivateDnsState);
} else {
validationLog("Network would not satisfy default request, "
+ "not validating");
transitionTo(mValidatedState);
}
return HANDLED;
}
mEvaluateAttempts++;
transitionTo(mProbingState);
return HANDLED;
If the network is a trusted one (e.g. WiFi selected manually by user, as opposed to WiFi selected by an App, for example), it would further transition into the ProbingState. The ProbingState is where the Samsung code derivates from the standard one.
Note the caution exercised by the original Android code, as explained in the comments above, before sending out the probe: e.g. the WiFi may be via an expensive metered network (e.g. tethering with a roaming phone). The frequent qq.com queries sent by Samsung’s WCM code, illustrated in Part 2, did not seem to take sufficient care of this by checking the trusted and metered flags.
The usual suspect
Before checking out how the probing is done, it is time to look at the usual suspect, the inChinaNetwork() method in the Network Monitor (line 2152-)
public boolean inChinaNetwork() {
if (this.mIsWifiOnly) {
updateCountryIsoCode();
if (isChineseIso(this.mScanResultsCountryIso) || isChineseIso(this.mDeviceCountryIso)) {
Log.d(TAG, "Wi-Fi Only CISO: " + this.mScanResultsCountryIso + "/" + this.mDeviceCountryIso);
return true;
}
Log.d(TAG, "Wi-Fi Only ISO: " + this.mScanResultsCountryIso + "/" + this.mDeviceCountryIso);
return false;
}
String str3 = this.mCountryIso;
if (str3 == null || str3.length() != 2) {
updateCountryIsoCode();
}
if (!isChineseIso(this.mCountryIso)) {
return false;
}
Log.d(TAG, "CISO: " + this.mCountryIso);
return true;
}
private boolean isChineseIso(String str) {
return "cn".equalsIgnoreCase(str) || "hk".equalsIgnoreCase(str) || "mo".equalsIgnoreCase(str);
}
The isChineseIso() method under NetworkMonitor, a different instance from the method of the same name as shown in Part 1 and the third of its kind discussed, applies the same logic, returning true for hk. Based on this method, the inChinaNetwork() method applies two different tests to determine if it is in a network in China, depending on whether the device is WiFi-only (e.g. some tablets) or not.
From the above, for WiFi-only devices, it would return true if either mScanResultsCountryIso or mDeviceCountryIso, determined by updateCountryIsoCode(), is regarded as a Chinese country code with isChineseIso(). For other devices, it would just look at mCountryIso.
The updateCountryIsoCode() method is listed below (line 2182-).
private void updateCountryIsoCode() {
TelephonyManager telephonyManager = mTelephonyManager;
if (telephonyManager != null) {
mTelephonyCountryIso = telephonyManager.getNetworkCountryIso();
Log.d(TAG, "updateCountryIsoCode() via TelephonyManager : ISO : " + mTelephonyCountryIso);
}
try {
mDeviceCountryIso = SemCscFeature.getInstance().getString("CountryISO");
Log.d(TAG, "updateCountryIsoCode() via SemCscFeature : ISO : " + mDeviceCountryIso);
} catch (Exception unused) {
}
if (mScanResultsCountryIso == null) {
mScanResultsCountryIso = mDependencies.getSetting(this.mContext, "wifi_wcm_country_code_from_scan_result", (String) null);
Log.d(TAG, "updateCountryIsoCode() via Scan Results : ISO : " + mScanResultsCountryIso);
}
if (mTelephonyCountryIso == null || str4.length() != 2) {
String str5 = mDeviceCountryIso;
if (str5 != null && str5.length() == 2) {
mCountryIso = mDeviceCountryIso;
}
} else {
mCountryIso = mTelephonyCountryIso;
}
Log.d(TAG, "updateCountryIsoCode() ISO : " + mCountryIso + " [" + mTelephonyCountryIso + "/" + mDeviceCountryIso + "/" + mScanResultsCountryIso + "]");
}
We can see that for WiFi-only device, the mScanResultsCountryIso used is from the winner of the WiFi country code scanning in WCM discussed earlier. This value is read only at the first time the method is called after NetworkMonitor is instantiated. If either this value or the mDeviceCountryIso, i.e. the firmware country code, equals cn, hk or mo, inChinaNetwork() will return true.
Recalling how the scan result is evaluated, this means that even if your firmware is not from Mainland China / Hong Kong, when you try to connect to a WiFi network, and you are in areas which, for whatever reason, have several WiFi access points broadcasting CN or HK (e.g. someone installed batches of inexpensive access points imported directly from China e-commerce sites), or your device has been passing through such an area (without connecting) in the past 24 hours, your device may regard itself to be in a China network.
For phone devices, the logic is the same as that in Part 1, i.e. the country is determined from the country code of the mobile network, and if that is not available, the device country code is used as the fallback.
Logic of the Probing State
Back to the ProbingState. On entry of this state, the following method is executed (line 849-）:
public void enter() {
if (NetworkMonitor.this.mEvaluateAttempts >= 5) {
TrafficStats.clearThreadStatsUid();
}
NetworkMonitor.this.mProbeToken ++;
if (!NetworkMonitor.this.mNetworkCapabilities.hasTransport(TRANSPORT_WIFI /*1*/) || !NetworkMonitor.this.inChinaNetwork()) {
mThread = new Thread(() -> {
NetworkMonitor.this.sendMessage(networkMonitor.obtainMessage(16, i, 0, networkMonitor.isCaptivePortal()));
});
} else {
if (NetworkMonitor.this.getRoamEventfromWCM()) {
NetworkMonitor.this.mRunFullParallelCheck = true;
Log.d(NetworkMonitor.TAG, "Wi-Fi Roam Event received from WCM. Run full parallel check");
NetworkMonitor.this.setRoamEventFromWCM(0);
}
mThread = new Thread(() -> {
NetworkMonitor.this.sendMessage(networkMonitor.obtainMessage(16, i, 0, networkMonitor.isCaptivePortalForChinaWifi()));
});
}
mThread.start();
}
getRoamEventfromWCM() is defined in line 2234–
private boolean getRoamEventfromWCM() {
return this.mDependencies.getSetting(this.mContext, "wifi_wcm_event_roam_complete", 0) == 1;
}
From the above code, if inChinaNetwork() returns true for a WiFi network, it will call isCaptivePortalForChinaWifi(), after setting the flag mRunFullParallelCheck – the flag would be set to false upon entry to the ValidState. But if getRoamEventfromWCM() returns true, i.e. after the Roam complete event is sent by the WCM via the wifi_wcm_event_roam_complete setting, mRunFullParallelCheck would be set to true.
Then we come to the interesting part, the isCaptivePortalForChinaWifi() method:
URLs of four popular Mainland Chinese sites are defined in line 78:
private static final String[] SECONDARY_HTTP_URLS_CHINA = {"http://www.qq.com", "http://www.baidu.com", "http://m.taobao.com", "http://m.hao123.com"};
isCaptivePortalForChinaWifi is defined at line 1820-:
public CaptivePortalProbeResult isCaptivePortalForChinaWifi() {
URL url, url2, url3;
CaptivePortalProbeResult captivePortalProbeResult;
if (!mIsCaptivePortalCheckEnabled) {
validationLog("Validation disabled.");
return CaptivePortalProbeResult.SUCCESS;
}
URL makeURL = makeURL("http://connectivity.samsung.com.cn/generate_204");
if (mRunFullParallelCheck) {
url2 = makeURL(SECONDARY_HTTP_URLS_CHINA[0]);
url = makeURL(SECONDARY_HTTP_URLS_CHINA[1]);
} else {
Random random = new Random(SystemClock.elapsedRealtime());
String[] strArr = SECONDARY_HTTP_URLS_CHINA;
int length = strArr.length;
int nextInt = random.nextInt(strArr.length);
url = makeURL(SECONDARY_HTTP_URLS_CHINA[(nextInt + 1) % length]);
url2 = makeURL(SECONDARY_HTTP_URLS_CHINA[nextInt % length]);
}
ProxyInfo httpProxy = mLinkProperties.getHttpProxy();
if (httpProxy == null || Uri.EMPTY.equals(httpProxy.getPacFileUrl())) {
url3 = null;
} else {
url3 = makeURL(httpProxy.getPacFileUrl().toString());
if (url3 == null) {
return CaptivePortalProbeResult.FAILED;
}
}
if (url3 == null && (makeURL == null || url2 == null || url == null)) {
return CaptivePortalProbeResult.FAILED;
}
long elapsedRealtime = SystemClock.elapsedRealtime();
if (url3 != null) {
captivePortalProbeResult = sendDnsAndHttpProbes(null, url3, 3);
reportHttpProbeResult(8, captivePortalProbeResult);
} else {
captivePortalProbeResult = sendParallelHttpProbesForChinaWifi(httpProxy, makeURL, url2, url, mRunFullParallelCheck);
}
long elapsedRealtime2 = SystemClock.elapsedRealtime();
sendNetworkConditionsBroadcast(true, captivePortalProbeResult.isPortal(), elapsedRealtime, elapsedRealtime2);
log("isCaptivePortalForChinaWifi: isSuccessful()=" + captivePortalProbeResult.isSuccessful() + " isPortal()=" + captivePortalProbeResult.isPortal() + " RedirectUrl=" + captivePortalProbeResult.redirectUrl + " isPartialConnectivity()=" + captivePortalProbeResult.isPartialConnectivity() + " Time=" + (elapsedRealtime2 - elapsedRealtime) + "ms");
return captivePortalProbeResult;
}
The first test is for mIsCaptivePortalCheckEnabled, which records the value of the setting captive_portal_mode at startup of the Monitor. Recall that in Samsung phones, this setting is sent from WCM and is mostly true.
Then the code will prepare a few URLs for probing. One is http://connectivity.samsung.com.cn/generate_204 which is Samsung China’s version of Google’s http://connectivitycheck.gstatic.com/generate_204. The 2nd and 3rd ones are two Mainland Chinese sites selected from the 4 sites under SECONDARY_HTTP_URLS_CHINA: if mRunFullParallelCheck is true (i.e. just after the roam complete event), these will be qq.com and baidu.com. Otherwise it can be any two of the four sites.
Another URL is obtained from ProxyInfo.getPacFileUrl() method which is from the Android API (refer to the developer documentation). See the wikipedia article for what PAC (Proxy Auto-Config) is. If this PAC URL is set for a WiFi access point, both the Samsung and the array of Mainland China URLs would be ignored when connecting to it, and instead probing will be done with the PAC URL specified, using sendDnsAndHttpProbes() below. The reason is explained in the standard Android code, largely due to the complexity of interpreting the PAC (which is JavaScript intended for browser consumption) at the system level.
Actual sending of probes
Finally let’s see how the actual probing is performed. The sendParallelHttpProbesForChinaWifi() is at line 1867-. The anonymous class in the decompiled code seems to be duplicated. But in essence, the core part is –
if (mType == 0) {
this.mResult = NetworkMonitor.this.sendDnsAndHttpProbes(proxy, httpUrl, 1);
} else if (mRunFullParallelCheck && mType == 1) {
this.mResult = NetworkMonitor.this.sendDnsAndHttpProbes(proxy, httpUrlCn, 1);
} else if (mRunFullParallelCheck && mType == 2) {
this.mResult = NetworkMonitor.this.sendDnsAndHttpProbes(proxy, httpUrlCn2, 1);
} else if (!mRunFullParallelCheck && mType == 1) {
this.mResult = NetworkMonitor.this.sendDnsAndCheckSocketSetup(proxy, httpUrlCn, 1);
} else if (!mRunFullParallelCheck && mType == 2) {
this.mResult = NetworkMonitor.this.sendDnsAndCheckSocketSetup(proxy, httpUrlCn2, 1);
}
mType controls which one of the three URLs are being probed (1 for the samsung.com.cn address, and 2 or 3 for the two China URL addresses respectively). In other words, the Samsung URL is probed using sendDnsAndHttpProbes(), while the two Chinese URLs are probed using sendDnsAndHttpProbes() when mRunFullParallelCheck is true (in which case they are always qq.com and baidu.com), and sendDnsAndCheckSocketSetup() otherwise.
In case mRunFullParallelCheck is true, sendDnsAndHttpProbes() may be called using samsung.com.cn, qq.com and baidu.com. It would proceed as follows (line 1205-):
private CaptivePortalProbeResult sendDnsAndHttpProbes(ProxyInfo proxyInfo, URL url, int i) {
return sendDnsAndHttpProbes(proxyInfo, url, i, null);
}
private CaptivePortalProbeResult sendDnsAndHttpProbes(ProxyInfo proxyInfo, URL url, int i, CaptivePortalProbeSpec captivePortalProbeSpec) {
InetAddress[] sendDnsProbe = sendDnsProbe(proxyInfo != null ? proxyInfo.getHost() : url.getHost());
if (!mCheckForDnsPrivateIpResponse || i == 3 || proxyInfo != null || mIgnorePrivateIpResponse || !isPrivateIpAddress(sendDnsProbe)) {
return sendHttpProbe(url, i, captivePortalProbeSpec);
}
return CaptivePortalProbeResult.PRIVATE_IP;
}
Unless proxyInfo was set (i.e. WiFi connection has a proxy), it will send DNS query for the provided URL through sendDnsProbe() at line 1222, which eventually calls the default system DNS resolver. The actual call is similar to standard Android process which can be traced here. Unlike the DNS resolver under Samsung’s custom code invoked by the WCM for qq.com as discussed in Part 2, the system DNS resolver would respect private DNS settings, and would make query to one of the DNS servers instead of all of them in parallel.
After getting the DNS results, if the result is not a private IP address (among other scenarios), it would call sendHttpProbe(), which performs the following (line 1302-, with comments from the corresponding Android source).
try {
// Follow redirects for PAC probes as such probes verify connectivity by fetching the
// PAC proxy file, which may be configured behind a redirect.
final boolean followRedirect = probeType == ValidationProbeEvent.PROBE_PAC;
final HttpURLConnection conn = (HttpURLConnection) mCleartextDnsNetwork.openConnection(url);
conn.setInstanceFollowRedirects(followRedirects);
conn.setConnectTimeout(SOCKET_TIMEOUT_MS); // 10000
conn.setReadTimeout(SOCKET_TIMEOUT_MS);
conn.setRequestProperty("Connection", "close");
conn.setUseCaches(false);
if (mCaptivePortalUserAgent != null) {
conn.setRequestProperty("User-Agent", mCaptivePortalUserAgent);
}
// cannot read request header after connection
String requestHeader = urlConnection.getRequestProperties().toString();
// Time how long it takes to get a response to our request
long requestTimestamp = SystemClock.elapsedRealtime();
httpResponseCode = urlConnection.getResponseCode();
redirectUrl = urlConnection.getHeaderField("location");
...
} catch (IOException e) {
validationLog(probeType, url, "Probe failed with exception " + e);
if (httpResponseCode == CaptivePortalProbeResult.FAILED_CODE /*599*/) {
// TODO: Ping gateway and DNS server and log results.
}
} finally {
if (urlConnection != null) {
urlConnection.disconnect();
}
TrafficStats.setThreadStatsTag(oldTag);
}
mCaptivePortalUserAgent, which was sent to the remote server with the HTTP request, defaults to "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.32 Safari/537.36" (line 1143-) which is the same as the standard Android source. Unless there are additional signals added to the actual HTTP request, this should not divulge additional information besides your IP address, that a device on that address probably is a recent Samsung phone, and the time pattern when the phone is actively connected via WiFi.
In the other case when mRunFullParallelCheck is false, sendDnsAndCheckSocketSetup() is called on 2 of the 4 Chinese sites. The difference with the earlier sendDnsAndHttpProbes(), as evident from its name, is that it just tries to establish a socket connection to the HTTP port (80) using the WiFi network.
public CaptivePortalProbeResult checkSocketSetup(URL url, int i, CaptivePortalProbeSpec captivePortalProbeSpec) {
...
int i2 = CaptivePortalProbeResult.FAILED_CODE/*599*/;
try {
long elapsedRealtime = SystemClock.elapsedRealtime();
Socket createSocket = this.mNetwork.getSocketFactory().createSocket(url.getHost(), 80);
long elapsedRealtime2 = SystemClock.elapsedRealtime();
i2 = CaptivePortalProbeResult.SUCCESS_CODE/*204*/;
...
validationLog(i, url, "time=" + j + "ms ret=" + 204 + " [checkSocketSetup] socket=" + createSocket.toString());
} catch (IOException e) {
validationLog(i, url, "checkSocketSetup failed with exception " + e);
}
...
if (captivePortalProbeSpec == null) {
return new CaptivePortalProbeResult(i2, null, url.toString());
}
return captivePortalProbeSpec.getResult(i2, null);
}
It appears the socket is not explicitly closed (unless there is error in decompilation). Anyway, although no application level information (like an HTTP request) is transmitted, it would be sufficient for the remote site to log your IP address, and the implication may be similar to the HTTP connection (although without the HTTP request and the Agent string, the server may be a bit less certain that this is a probe from Samsung phones).
Following the logic of sendParallelHttpProbesForChinaWifi() at line 2037-, when mRunFullParallelCheck is true (occurring once after WiFi roaming is completed), it would probe samsung.com.cn first. If connection is successful or a captive portal is found (a site returning 200), it would return the findings. Otherwise (perhaps while the samsung.com.cn site is temporarily reachable), it would probe the two Chinese websites in parallel. If both of these website redirects to the same domain, it is considered to be in a captive portal. On the other hand, when mRunFullParallelCheck is false, the probe to two of the Chinese sites is just for checking connectivity, and may not return sufficient information to tell a captive portal site apart from a successful connection to the Internet.
if (mRunFullParallelCheck) {
try {
r13.start(); // r13 is the probe for samsung.com.cn
countDownLatch.await(3000, TimeUnit.MILLISECONDS);
} catch (InterruptedException unused) {
validationLog("Error: probes wait interrupted!");
return CaptivePortalProbeResult.FAILED;
}
}
CaptivePortalProbeResult result = r13.result();
if (result.isPortal()) {
reportHttpProbeResult(8, result);
return result;
} else if (result.isSuccessful()) {
reportHttpProbeResult(16, result);
return result;
} else if (!mCheckForDnsPrivateIpResponse || !result.isDnsPrivateIpResponse()) {
try {
r14.start(); // r14 and r15 is the probe for 2 Chinese sites
r15.start();
countDownLatch2.await(3000, TimeUnit.MILLISECONDS);
CaptivePortalProbeResult result2 = r14.result();
CaptivePortalProbeResult result3 = r15.result();
if (mRunFullParallelCheck && result2.isPortal() && result3.isPortal()) { // isPortal() means status code 200
if (!(result2.redirectUrl == null || result3.redirectUrl == null)) {
try {
if (new URL(str).getHost().equals(new URL(str2).getHost())) {
Log.d(TAG, "RDTTSL - " + url4.getHost());
reportHttpProbeResult(32, result2);
return result2;
}
} catch (MalformedURLException e) {
Log.e(TAG, "sendParallelHttpProbesForChinaWifi MalformedURLException: " + e);
}
}
}
if (mCheckForDnsPrivateIpResponse && (result2.isDnsPrivateIpResponse() || result3.isDnsPrivateIpResponse())) {
validationLog("DNS response to the URL is private IP - CHINA WIFI");
return CaptivePortalProbeResult.FAILED;
} else if (result2.mHttpResponseCode == CaptivePortalProbeResult.FAILED_CODE
&& result3.mHttpResponseCode == CaptivePortalProbeResult.FAILED_CODE) {
// If both Chinese sites failed, it would return the samsung.com.cn result
try {
r13.join();
reportHttpProbeResult(8, r13.result());
reportHttpProbeResult(16, r13.result());
return r13.result();
} catch (InterruptedException unused2) {
validationLog("Error: http or https probe wait interrupted!");
return CaptivePortalProbeResult.FAILED;
}
} else {
CaptivePortalProbeResult captivePortalProbeResult = new CaptivePortalProbeResult(CaptivePortalProbeResult.SUCCESS_CODE/*204*/, null,
result2.mHttpResponseCode != CaptivePortalProbeResult.FAILED_CODE ? result2.detectUrl : result3.detectUrl);
// 204 above denotes successful, if any one of the two Chinese site probe is successful.
reportHttpProbeResult(16, captivePortalProbeResult);
return captivePortalProbeResult;
}
} catch (InterruptedException unused3) {
validationLog("Error: probes wait interrupted! - CHINA WIFI");
return CaptivePortalProbeResult.FAILED;
}
} else {
validationLog("DNS response to the URL is private IP - CHINA WIFI");
return CaptivePortalProbeResult.FAILED;
}
The “Normal” case
In the alternative case of inChinaNetwork() returning false, it will perform the probe by calling isCaptivePortal(). This is similar to the standard Android handling, i.e. largely respecting the setting under the Global Settings captive_portal_https_url and captive_portal_http_url to set the query site, and respect the captive_portal_use_https flag (which defaults to true) which requires that one of the probes to use HTTPS, such that a private network cannot impersonate as the test site such as Google, causing the device to report as connected to Internet when in fact it is not. (In contrast, note that no HTTPS probes are used when inChinaNetwork() is true above, as all the hardcoded addresses use HTTP. In other words, this security feature, which is on by default, is disabled for Samsung phones in Hong Kong, Macau and Mainland China.)
The workaround
Recall that if the PAC URL is present in the Proxy setting of the WiFi access point concerned, test for connectivity would be done with the PAC URL instead. Thus, while the standard captive_portal_mode global setting cannot be used in Samsung phones to disable the probes with Samsung or Mainland China sites as discussed in this article, a simple means of workaround to stop all such probes is as follows.
Set the PAC URL for the WiFi connection concerned to a URL serving a PAC file (for an access point, choose advanced settings, set the proxy option to Proxy Auto Configure, and enter the URL). To avoid actually affecting the browsers, the URL should return a PAC which does nothing, e.g. [4]
function FindProxyForURL(url, host) { return "DIRECT"; }To save data and avoid setting up a URL, if one does not need to use public WiFi / captive portals, an alternative is to run a simple App to serve the PAC locally, by running, for example, this open-sourced PAC Service App developed by the author, and setting the PAC URL to point to the localhost URL provided. Apk file is also available at the download page of this site.
But please note that the setting has no effect on the DNS queries to qq.com sent by the WCM as discussed in Part 2. To stop those one need to change the SSID to match the patterns as discussed, until the issue is fixed in the firmware.
[1] The default fallback in standard Android are http://connectivitycheck.gstatic.com/generate_204 and https://www.google.com/generate_204 under NetworkStackUtils.java, if the setting is absent. For Samsung WiFi-only devices, the URL for the HTTPS test site is hardcoded to https://connectivitycheck.gstatic.com/generate_204 at line 422.
[2] This version has already updated its handling of DNS server as described in Part 1, and force the DNS 114.114.114.114 only on China version of its firmware, but it has not touched the issue covered in Part 2.
[3] The delay is controlled by mValidationCheckTime, but strangely it divides by 2 in each loop, increasing the checking frequency i.e. the opposite of the usual exponential backoff, before the loop count reachs VALIDATION_CHECK_MAX_COUNT, defined as 4.
[4] The author has set up a sample Github gist with the URL https://gist.githubusercontent.com/headuck/809b0f793ae2cc9b88d2d44362d03a95/raw/729b26145df9eeb3cac3ffae7d55f2a950251870/direct.pac as an example. But this link cannot be guarenteed to persist.
在〈How Samsung devices connect WiFi users in Hong Kong and possibly Worldwide to Tencent, Baidu, Taobao and Hao123〉中有 1 則留言
Now, I can only find Call Blocker on the download page, can you release the PAC Service App again? I wish I could package the code.