I read a bit of code and thus replying to my own message.
It seems rpkg decides whether to upload a file to the lookaside cache or not based on running "file -b --mime-encoding" on it. For a keyring file that returns "binary". See https://pagure.io/rpkg/blob/master/f/pyrpkg/utils.py#_280
-Ville-Pekka