Loosen requirement for `object` in media upload

[evanp]

Not entirely related to the point @reiver is trying to make, so apologies for piggybacking on this ticket.

If changes are still up for consideration, I'd like to propose a slight simplification to the MediaUpload extension as it's presented in the above SocialCG link.

Instead of pushing both binary content and the "shell" ActivityPub Create activity in which to slot the uploaded media, the server could handle strictly the media upload, and return an incomplete ActivityPub Object (a "shell" as it was called) containing the properties that have been inferred from the uploaded media:

• Type: Audio, Video, Image, etc.
• MediaType: audio/..., video/..., image/..., etc.
• URL: containing the direct link to the uploaded media.
• AttributedTo: inferred from the authorization the upload was effected under, if it refers to a valid user
• etc

Then the client the user used for the upload can append the additional elements the user specifies (Summary, Name, ID, etc) and use a Create activity to actually create the ActivityPub object.

Additionally this allows for servers that operate fancier upload mechanisms (eg, creating different web-optimized versions for the data, or, as @reiver asked, a server that can support partial uploads) but still represent them as valid ActivityPub objects.

To me, this is a better API as it separates the concerns between the actual media upload and the ActivityPub processing, and in my case, the reason why I like this architecture better is that it allows for the mediaUpload to be a separate micro-service, independent of the ActivityPub server.

Originally posted by @mariusor in #578

[evanp]

So, the easiest possible way to do this I can think of is, not returning an ActivityPub object id, but just the location of the uploaded file. Something like this:

POST /endpoints/mediaUpload HTTP/1.1
Content-Type: image/png

...

And a result like this:

HTTP/1.1 204 No Content
Location: https://media.social.example/files/uploaded-file-33718.png

Which could be used with a POST to the outbox like this:

{
   "@context": "https://www.w3.org/ns/activitystreams",
   "type": "Create",
   "to": "as:Public",
   "object": {
       "to": "as:Public",
       "type": "Image",
       "url": {
           "to": "as:Public",
           "type": "Link",
           "href":  "https://media.social.example/files/uploaded-file-33718.png"
       }
   }
}

I think this would require some smarts on the server side to detect that this is an uploaded file, that it has not previously been Created, and that it is now available to the Public. But it's not an overwhelming requirement.

I think this is not much more of a burden than the multipart/form-data format, and might be a little easier to manage the upload side.

[evanp]

I think the big difference is that this interaction is not atomic; you have the upload and the create in two different requests. There's a lot of opportunities for error states, like never sending the Create activity, or sending more than one, or sending it from the wrong account, dot dot dot.

[evanp]

We had a big discussion about whether the URL in the above example should just be a string. I think this is problematic as discussed here: w3c/activitystreams#498 however, we did not have consensus on that. I still think the above syntax should work OK.

[evanp]

There was also some concern during issue triage about requiring access control on the uploaded file; for example, doing a private or followers-only Create activity. I think that's unrelated to the separation of the JSON-LD from the file upload, but still worth noting.

[trwnh]

My concern is that the described flow in #579 (comment) is both indirect and retroactively affects <https://media.social.example/files/uploaded-file-33718.png>. If it should affect <https://media.social.example/files/uploaded-file-33718.png> then it should be a direct reference so that the metadata can be directly applied. In other words, the requirement in AP of copying addressing from the activity to the object and vice-versa needs to be extended to also cover... "the object.url.href if it has never been seen in any prior activity"?

That's described as "not an overwhelming requirement", but it does seem pretty whelming to me!

It also creates a problem that any "first use" of an uploaded media asset will invariably exhaust it. If you use it in an activity addressed to the Public, your media is now Public. If you use it in an activity addressed to your followers, your media is now followers-only? So if you want to use it in a Public activity later, you have to reupload the same media asset (and good luck deduplicating them later, I guess!)

I think if you want to attach concerns to the media, it should be done directly at the time of upload, as described in https://www.w3.org/wiki/SocialCG/ActivityPub/MediaUpload where the multipart media typing is used to provide the metadata and the binary blob at the same time.

In fact, the proposal by @mariusor preserves this quality, although it seems to propose inferring properties based on media file's embedded metadata.

POST /media/ HTTP/1.1
Content-Type: image/png

...

HTTP/1.1 201 Created
Location: /media/1
Content-Type: application/activity+json
Comment: This could also be a 202 Accepted if async media processing is needed (large video files, etc).

{
  "id": "/media/1",
  "type": "Image",
  "url": [
    {"href": "/media/1/original", "mediaType": "image/png"}, // this is what you uploaded
    {"href": "/media/1/compressed", "mediaType": "image/jpg"} // this could be auto-converted from original
  ]
}

So instead of multipart content, you upload just the media, but the Location still refers to the AS2 description for the Image object and not directly to the asset/blob that is the "image representation".

GET /media/1 HTTP/1.1

HTTP/1.1 200 OK
Content-Type: application/activity+json

{
  "id": "/media/1",
  "type": "Image",
  "url": [
    {"href": "/media/1/original", "mediaType": "image/png"},
    {"href": "/media/1/compressed", "mediaType": "image/jpg"}
  ]
}

This avoids the weird indirection through as:url/as:href.

Basically, we can summarize it like this:

• Original mediaUpload proposal: "here's a shell and some media -- stuff the media into the shell and return 202 Accepted; then have that activity be published to my outbox when media processing is done". So in this case, the mediaUpload endpoint is like a variant of the outbox, because it eventually results in an activity being published (with no further action on your behalf).
• Marius's proposal: "here's some media -- generate a shell based on its metadata and return the shell as the content of a 201 Created; then refer to that object as you would any other thing (possibly including additional information)". So in this case, the mediaUpload endpoint is like a custom outbox that accepts media instead of activities and returns objects instead of publishing activities. (I honestly think it could even be a Collection of all your uploaded media...)
• Evan's proposal: "here's some media -- give me the URL for the blob; then extend the outbox to introspect any Create of a media-ish object whose url.href is a never-before seen identifier, and retroactively update that resource if you can". So in this case, the mediaUpload endpoint is kind of like a basic HTTP PUT, except it assigns an identifier.

I don't see why retroactive updates of an object.url.href is needed as a mechanism, and it could very well be the case that the media is not hosted by the same server as the outbox -- but you'd still have to check every time.

[mariusor]

I just want to thank @evanp for submitting this on my behalf, and mention that the reason why I didn't do it myself was that it wasn't clear to me that there are new proposals accepted for the feature. My initial comment under a different ticket was mostly an inquiry about suitability with a big picture overview.

That being said, personally I prefer the simplest option: upload media, receive Location with its URL that can be used to create the activity object.