[Storage] upload roundtrip concerns with download API#4075
Draft
vincenttran-msft wants to merge 2 commits intoAzure:mainfrom
Draft
[Storage] upload roundtrip concerns with download API#4075vincenttran-msft wants to merge 2 commits intoAzure:mainfrom
upload roundtrip concerns with download API#4075vincenttran-msft wants to merge 2 commits intoAzure:mainfrom
Conversation
vincenttran-msft
commented
Apr 1, 2026
Comment on lines
+1182
to
+1196
| // Act — download the blob | ||
| // `download` returns AsyncResponse<BlobClientDownloadResult>: the body is a PinnedStream. | ||
| // PinnedStream is a Pin<Box<dyn Stream<Item=Result<Bytes>> + Send>>: it is forward-only | ||
| // and cannot be reset or asked for its length without consuming it first. | ||
| let download_response = source_blob_client.download(None).await?; | ||
|
|
||
| // The content-length header is available, but the body stream itself has no `len()` method | ||
| // and no `reset()`. There is no conversion from AsyncResponseBody into RequestContent or | ||
| // Body::SeekableStream. The only path forward is to collect the entire stream into memory. | ||
| let buffered: Bytes = download_response.into_body().collect().await?; | ||
|
|
||
| // Only after full buffering can the data be passed to `upload`. | ||
| dest_blob_client | ||
| .upload(RequestContent::from(buffered.to_vec()), None) | ||
| .await?; |
Member
Author
There was a problem hiding this comment.
This is the crux of the issue.
vincenttran-msft
commented
Apr 1, 2026
Comment on lines
+1340
to
+1346
| let stream = DownloadBodySeekableStream { | ||
| inner: Arc::new(Mutex::new(DownloadBodyInner { | ||
| stream: Some(Box::pin(body)), | ||
| remainder: Bytes::new(), | ||
| })), | ||
| len: content_length.unwrap_or(0), | ||
| }; |
Member
Author
There was a problem hiding this comment.
This is the crux of the workaround: a custom DownloadBodySeekableStream adapter so that we can get it into a RequestContent without collecting in-memory.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is applicable to its current state, but will also still be applicable even after
managed_downloadreplacesdownload.Against our current authoring, the only way to upload a downloaded blob is to buffer in-memory to be able to get it into a
RequestContent:Applying a similar workaround as @benbp :