Migrate Base64 away from Msebera (in Android), Fixes AB#3077784 (#2389)

[AB#3077784](https://identitydivision.visualstudio.com/fac9d424-53d2-45c0-91b5-ef6ba7a6bf26/_workitems/edit/3077784)
(Part 2)

## Background

Our entire code base is written to be compatible with
Android.util.Base64, which is a fork of a legacy version of apache
commons-codec.

This means Android OS ships with apache commons-codec, and we cannot
have that as a dependency in common4j project (otherwise the classloader
would throw an error).

The safest, previous workaround is to consume [msebera
HttpClient](https://github.com/smarek/httpclient-android), which also
contains commons-codec, but under a different namespace. [(This is
officially suggested by Apache)
](https://hc.apache.org/httpcomponents-client-4.5.x/android.html)

## Issue 

msebera library is no longer maintained, and it was recently flagged
with an apk tool.

Even if the flagged part is NOT the code path we use. It's better that
we migrate away from it (so that our customer don't see the false alarm)

There are multiple ways to tackle this
1. (Least risky) Make the android project consume Android.util.base64,
keep msebera for Linux (which is already in maintainance mode) and test
classes
2. Migrate to other libraries. (e.g. guava)
   - NOTE: Java's Base64 is not an option because it requires min API 26

I decide to go with 1 because 
- the other option might increase the final apk/aar size by a lot. 
- It would also introduce a breaking change (e.g. The value in storage -
before upgrade - was encoded with msebera, and might not be compatible
with the new library.
- The new library might also handle error cases/unexpected format
differently.

## Changes

All the Base64 usage is now consolidated into the Base64Util class.
(most of the changes are under this bucket).

This Base64Util class will use a classloader to pick the implementation
to use.
- In Android modules, it would load `AndroidBase64`
- In Linux Broker, it would load `MseberaBase64ForLinux`
- In Common4j's own unit test, it would load
`MseberaBase64ForCommon4jTests` (from common4j's testImplementation)
- In Broker4j's own unit test, it would load
`MseberaBase64ForBroker4jTests` (from common4j's testImplementation)
- In other tests, it would load `MseberaBase64ForTestUtils` (from
:testutils)

Author: rpdome

changelog.txt

@@ -1,5 +1,6 @@
 vNext
----------
+----------
+- [MINOR] Migrate Base64 away from Msebera httpclient (#2389)
 - [PATCH] Provide empty image when no video is present on QR +PIN auth (#2424)
 - [MINOR] On GetPreferredAuthMethod return NONE when Authenticator is not installed (#2523)
 - [MINOR] Migrate URIbuilder to httpcore5 (#2522)

common/build.gradle

@@ -156,7 +156,6 @@ dependencies {
     implementation(group: 'com.microsoft.device.display', name: 'display-mask', version: '0.3.0')
 
     implementation 'org.apache.httpcomponents.core5:httpcore5:5.3'
-    implementation "cz.msebera.android:httpclient:$rootProject.ext.mseberaApacheHttpClientVersion"
     implementation "com.nimbusds:nimbus-jose-jwt:$rootProject.ext.nimbusVersion"
     implementation "androidx.appcompat:appcompat:$rootProject.ext.appCompatVersion"
     implementation "com.google.code.gson:gson:$rootProject.ext.gsonVersion"

common/src/main/java/com/microsoft/identity/common/adal/internal/util/StringExtensions.java

@@ -26,6 +26,8 @@
 import android.util.Base64;
 import android.util.Log;
 
+import com.microsoft.identity.common.java.base64.Base64Flags;
+import com.microsoft.identity.common.java.base64.Base64Util;
 import com.microsoft.identity.common.java.exception.ErrorStrings;
 import com.microsoft.identity.common.logging.Logger;
 
@@ -112,20 +114,6 @@ public static String urlFormDecode(String source) throws UnsupportedEncodingExce
         return URLDecoder.decode(source, ENCODING_UTF8);
     }
 
-    /**
-     * Encode Base64 URL Safe String.
-     *
-     * @param bytes byte[]
-     * @return String
-     * @throws UnsupportedEncodingException throws if encoding not supported.
-     */
-    public static String encodeBase64URLSafeString(final byte[] bytes)
-            throws UnsupportedEncodingException {
-        return new String(
-                Base64.encode(bytes, Base64.NO_PADDING | Base64.NO_WRAP | Base64.URL_SAFE),
-                ENCODING_UTF8);
-    }
-
     /**
      * create url from given endpoint. return null if format is not right.
      *
@@ -240,16 +228,6 @@ public static boolean hasPrefixInHeader(final String value, final String prefix)
                 && Character.isWhitespace(value.charAt(prefix.length()));
     }
 
-    /**
-     * Based64URL encode the input string.
-     *
-     * @param message String
-     * @return String
-     */
-    public static String base64UrlEncodeToString(final String message) {
-        return Base64.encodeToString(message.getBytes(Charset.forName(ENCODING_UTF8)), Base64.URL_SAFE | Base64.NO_WRAP);
-    }
-
     /**
      * Append parameter to the url. If the no query parameters, return the url originally passed in.
      */

common/src/main/java/com/microsoft/identity/common/base64/AndroidBase64.kt

@@ -0,0 +1,63 @@
+// Copyright (c) Microsoft Corporation.
+// All rights reserved.
+//
+// This code is licensed under the MIT License.
+//
+// Permission is hereby granted, free of charge, to any person obtaining a copy
+// of this software and associated documentation files(the "Software"), to deal
+// in the Software without restriction, including without limitation the rights
+// to use, copy, modify, merge, publish, distribute, sublicense, and / or sell
+// copies of the Software, and to permit persons to whom the Software is
+// furnished to do so, subject to the following conditions :
+//
+// The above copyright notice and this permission notice shall be included in
+// all copies or substantial portions of the Software.
+//
+// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+// THE SOFTWARE.
+package com.microsoft.identity.common.base64
+
+import com.microsoft.identity.common.java.base64.Base64Flags
+import com.microsoft.identity.common.java.base64.Base64Util
+import com.microsoft.identity.common.java.base64.IBase64
+
+/**
+ * Base64 implementation based on android.util.Base64
+ *
+ * If you need to rename or change the namespace of this class,
+ * you'll need to make change in [Base64Util] too.
+ *
+ * see [Base64Util] for more info.
+ **/
+
+class AndroidBase64 : IBase64 {
+
+    override fun encode(input: ByteArray, vararg flags: Base64Flags): ByteArray {
+        return android.util.Base64.encode(input, combineFlags(*flags))
+    }
+
+    override fun decode(input: ByteArray, vararg flags: Base64Flags): ByteArray {
+        return android.util.Base64.decode(input, combineFlags(*flags))
+    }
+
+    private fun combineFlags(vararg flags: Base64Flags): Int {
+        var combinedFlag = android.util.Base64.DEFAULT
+
+        if (flags.contains(Base64Flags.URL_SAFE)) {
+            combinedFlag = combinedFlag or android.util.Base64.URL_SAFE
+        }
+        if (flags.contains(Base64Flags.NO_WRAP)) {
+            combinedFlag = combinedFlag or android.util.Base64.NO_WRAP
+        }
+        if (flags.contains(Base64Flags.NO_PADDING)) {
+            combinedFlag = combinedFlag or android.util.Base64.NO_PADDING
+        }
+
+        return combinedFlag
+    }
+}

common/src/main/java/com/microsoft/identity/common/internal/cache/SharedPreferencesFileManager.java

@@ -24,7 +24,6 @@
 
 import android.content.Context;
 import android.content.SharedPreferences;
-import android.os.Build;
 import android.util.LruCache;
 
 import androidx.annotation.GuardedBy;

common/src/test/java/com/microsoft/identity/common/MicrosoftStsAccountCredentialAdapterTest.java

@@ -22,8 +22,9 @@
 // THE SOFTWARE.
 package com.microsoft.identity.common;
 
-import com.microsoft.identity.common.java.AuthenticationConstants;
 import com.microsoft.identity.common.java.authorities.Authority;
+import com.microsoft.identity.common.java.base64.Base64Flags;
+import com.microsoft.identity.common.java.base64.Base64Util;
 import com.microsoft.identity.common.java.cache.MicrosoftStsAccountCredentialAdapter;
 import com.microsoft.identity.common.java.commands.parameters.TokenCommandParameters;
 import com.microsoft.identity.common.java.dto.CredentialType;
@@ -49,21 +50,14 @@
 
 import org.junit.Before;
 import org.junit.Test;
-import org.junit.runner.RunWith;
 import org.mockito.Mockito;
-import org.powermock.core.classloader.annotations.PowerMockIgnore;
-import org.powermock.modules.junit4.PowerMockRunner;
 
 import java.net.MalformedURLException;
 import java.net.URL;
 import java.security.SecureRandom;
 import java.util.Calendar;
 import java.util.Date;
 import java.util.GregorianCalendar;
-import java.util.HashSet;
-import java.util.Set;
-
-import cz.msebera.android.httpclient.extras.Base64;
 
 import static com.microsoft.identity.common.java.providers.microsoft.MicrosoftAccount.AUTHORITY_TYPE_MS_STS;
 import static org.junit.Assert.assertEquals;
@@ -292,6 +286,6 @@ public void testCreateIdTokenRecord() {
 
     static String createRawClientInfo(final String uid, final String utid) {
         final String claims = "{\"uid\":\"" + uid + "\",\"utid\":\"" + utid + "\"}";
-        return Base64.encodeToString(StringUtil.toByteArray(claims), Base64.URL_SAFE);
+        return Base64Util.encodeToString(StringUtil.toByteArray(claims), Base64Flags.URL_SAFE);
     }
 }

common4j/build.gradle

@@ -218,9 +218,10 @@ dependencies {
     implementation "com.google.code.gson:gson:$rootProject.ext.gsonVersion"
     implementation "org.json:json:$rootProject.ext.jsonVersion"
     implementation "com.github.stephenc.jcip:jcip-annotations:$rootProject.ext.jcipAnnotationVersion"
-    implementation "cz.msebera.android:httpclient:$rootProject.ext.mseberaApacheHttpClientVersion"
     implementation 'org.apache.httpcomponents.core5:httpcore5:5.3'
 
+    testImplementation "cz.msebera.android:httpclient:4.5.8"
+
     testCompileOnly "com.github.spotbugs:spotbugs-annotations:$rootProject.ext.spotBugsAnnotationVersion"
     testCompileOnly "org.projectlombok:lombok:$rootProject.ext.lombokVersion"
     testAnnotationProcessor "org.projectlombok:lombok:$rootProject.ext.lombokVersion"

common4j/src/main/com/microsoft/identity/common/java/authorities/AuthorityDeserializer.java

@@ -29,14 +29,12 @@
 import com.google.gson.JsonParseException;
 import com.microsoft.identity.common.java.logging.Logger;
 import com.microsoft.identity.common.java.util.CommonURIBuilder;
+import com.microsoft.identity.common.java.util.StringUtil;
 
 import net.jcip.annotations.Immutable;
 
 import java.lang.reflect.Type;
 import java.net.URI;
-import java.util.List;
-
-import cz.msebera.android.httpclient.util.TextUtils;
 
 @Immutable
 public class AuthorityDeserializer implements JsonDeserializer<Authority> {
@@ -65,7 +63,7 @@ public Authority deserialize(JsonElement json, Type typeOfT, JsonDeserialization
                             final CommonURIBuilder uri = new CommonURIBuilder(URI.create(aadAuthority.mAuthorityUrlString));
                             final String cloudUrl = uri.getScheme() + "://" + uri.getHost();
                             final String tenant = uri.getLastPathSegment();
-                            if (!TextUtils.isEmpty(tenant)) {
+                            if (!StringUtil.isNullOrEmpty(tenant)) {
                                 aadAuthority.mAudience = AzureActiveDirectoryAudience.getAzureActiveDirectoryAudience(cloudUrl, tenant);
                             }
                         } catch (final IllegalArgumentException e) {

common4j/src/main/com/microsoft/identity/common/java/base64/Base64Flags.kt

@@ -0,0 +1,54 @@
+// Copyright (c) Microsoft Corporation.
+// All rights reserved.
+//
+// This code is licensed under the MIT License.
+//
+// Permission is hereby granted, free of charge, to any person obtaining a copy
+// of this software and associated documentation files(the "Software"), to deal
+// in the Software without restriction, including without limitation the rights
+// to use, copy, modify, merge, publish, distribute, sublicense, and / or sell
+// copies of the Software, and to permit persons to whom the Software is
+// furnished to do so, subject to the following conditions :
+//
+// The above copyright notice and this permission notice shall be included in
+// all copies or substantial portions of the Software.
+//
+// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+// THE SOFTWARE.
+package com.microsoft.identity.common.java.base64
+
+/**
+ * Flags for Base64 encoding/decoding options.
+ **/
+enum class Base64Flags {
+
+    /**
+     * Default values for encoder/decoder flags.
+     */
+    DEFAULT,
+
+    /**
+     * Encoder flag bit to omit the padding '=' characters at the end
+     * of the output (if any).
+     */
+    NO_PADDING,
+
+    /**
+     * Encoder flag bit to omit all line terminators (i.e., the output
+     * will be on one long line).
+     */
+    NO_WRAP,
+
+    /**
+     * Encoder/decoder flag bit to indicate using the "URL and
+     * filename safe" variant of Base64 (see RFC 3548 section 4) where
+     * `-` and `_` are used in place of `+` and
+     * `/`.
+     */
+    URL_SAFE
+}