Base 64 - padding and URLs

RFC 4648 lists two common variants:

• Base 64 encoding using alphanumeric chars, +, /, and = as the padding character.

• Base 64 encoding with URL and Filename Safe Alphabet using alphanumeric chars, -, _, and also = as the padding character.

Notice that both of them define = for padding.

The thing is that = is not really safe to use in URLs so you either need to encode it (additional hassle) or leave it out (and make sure you don’t break decoding). The RFC suggests that:

The pad character "=" is typically percent-encoded when used in an URL.

Hence I asked, in the pull request comment, if we should include = too.

However, section 3.2 also says this:

In some circumstances, the use of padding ("=") in base-encoded data is not required or used. In the general case, when assumptions about the size of transported data cannot be made, padding is required to yield correct decoded data.

In fact, as the Api Security in Action book tells us, it’s common to exclude padding because you often know the whole encoded value before the decoding process starts.

The book contains this class for encoding/decoding:

public class Base64Url {

    // For more about padding and when it's needed see: https://stackoverflow.com/questions/4080988/why-does-base64-encoding-require-padding-if-the-input-length-is-not-divisible-by
    private static final Base64.Encoder encoder = Base64.getUrlEncoder().withoutPadding();
    private static final Base64.Decoder decoder = Base64.getUrlDecoder();

    public static String encode(byte[] data) {
        // Note that this uses ISO-8859-1 - should be safe since Base64 uses only ASCII characters anyway
        return encoder.encodeToString(data);
    }

    public static byte[] decode(String encoded) {
        return decoder.decode(encoded);
    }

    public static void main(String[] args) {
        System.out.println("Default charset: " + Charset.defaultCharset());
        System.out.println(encode("ahojľščáýíô".getBytes()));
        System.out.println("decoded: " + new String(decode(encode("ahojľščáýíô".getBytes()))));
    }
}

Base 64 with Clojure and Buddy Library

In our code, we are using buddy.core.codecs/bytes→b64u. Let’s look at an example:

(-> "abc1230901" .getBytes codecs/bytes->b64u String
;;=> "YWJjMTIzMDkwMQ"

(-> "abc1230901" .getBytes codecs/bytes->b64 String.)
;;=> "YWJjMTIzMDkwMQ=="

(defn bytes->b64u
  "Encode data to base64 byte array (using url-safe variant)."
  {:added "1.8.0"}
  [^bytes data]
  (let [^Base64$Encoder encoder (-> (java.util.Base64/getUrlEncoder)
                                    (.withoutPadding))] ;; (1)
    (.encode encoder data)))