Airbyte S3 Connector on Kubernetes

This post was originally published on Pure Storage

<span style=”font-weight: 400;”>$ aws s3api list-objects-v2 –bucket airbytedest –profile fbstaines03 –</span><span style=”font-weight: 400;”>no</span><span style=”font-weight: 400;”>-verify-ssl –endpoint-url=https:</span><span style=”font-weight: 400;”>//192.168.40.165

<span style=”font-weight: 400;”> InsecureRequestWarning,</span>

<span style=”font-weight: 400;”>{</span>

<span style=”font-weight: 400;”>   </span><span style=”font-weight: 400;”>”Contents”</span><span style=”font-weight: 400;”>: [</span>

<span style=”font-weight: 400;”>       {</span>

<span style=”font-weight: 400;”>           </span><span style=”font-weight: 400;”>”LastModified”</span><span style=”font-weight: 400;”>: </span><span style=”font-weight: 400;”>”2023-08-08T11:56:47.000Z”</span><span style=”font-weight: 400;”>,</span>

<span style=”font-weight: 400;”>           </span><span style=”font-weight: 400;”>”ETag”</span><span style=”font-weight: 400;”>: </span><span style=”font-weight: 400;”>”0a08d7a59360c0ec2f53cd572ca86127-20″</span><span style=”font-weight: 400;”>,</span>

<span style=”font-weight: 400;”>           </span><span style=”font-weight: 400;”>”StorageClass”</span><span style=”font-weight: 400;”>: </span><span style=”font-weight: 400;”>”STANDARD”</span><span style=”font-weight: 400;”>,</span>

<span style=”font-weight: 400;”>           </span><span style=”font-weight: 400;”>”Key”</span><span style=”font-weight: 400;”>: </span><span style=”font-weight: 400;”>”/nyctaxi/fbS3nyctaxi/2023_08_08_1691495559589_0.avro”</span><span style=”font-weight: 400;”>,</span>

<span style=”font-weight: 400;”>           </span><span style=”font-weight: 400;”>”Size”</span><span style=”font-weight: 400;”>: </span><span style=”font-weight: 400;”>209738278</span>

<span style=”font-weight: 400;”>       },</span>

<span style=”font-weight: 400;”>       {</span>

<span style=”font-weight: 400;”>           </span><span style=”font-weight: 400;”>”LastModified”</span><span style=”font-weight: 400;”>: </span><span style=”font-weight: 400;”>”2023-08-08T12:00:51.000Z”</span><span style=”font-weight: 400;”>,</span>

<span style=”font-weight: 400;”>           </span><span style=”font-weight: 400;”>”ETag”</span><span style=”font-weight: 400;”>: </span><span style=”font-weight: 400;”>”3c81c74aa12620ada225390959677eb7-20″</span><span style=”font-weight: 400;”>,</span>

<span style=”font-weight: 400;”>           </span><span style=”font-weight: 400;”>”StorageClass”</span><span style=”font-weight: 400;”>: </span><span style=”font-weight: 400;”>”STANDARD”</span><span style=”font-weight: 400;”>,</span>

<span style=”font-weight: 400;”>           </span><span style=”font-weight: 400;”>”Key”</span><span style=”font-weight: 400;”>: </span><span style=”font-weight: 400;”>”/nyctaxi/fbS3nyctaxi/2023_08_08_1691495559589_1.avro”</span><span style=”font-weight: 400;”>,</span>

<span style=”font-weight: 400;”>           </span><span style=”font-weight: 400;”>”Size”</span><span style=”font-weight: 400;”>: </span><span style=”font-weight: 400;”>209747517</span>

<span style=”font-weight: 400;”>       },</span>

<span style=”font-weight: 400;”>       {</span>

<span style=”font-weight: 400;”>           </span><span style=”font-weight: 400;”>”LastModified”</span><span style=”font-weight: 400;”>: </span><span style=”font-weight: 400;”>”2023-08-08T12:04:56.000Z”</span><span style=”font-weight: 400;”>,</span>

<span style=”font-weight: 400;”>           </span><span style=”font-weight: 400;”>”ETag”</span><span style=”font-weight: 400;”>: </span><span style=”font-weight: 400;”>”f62110ec469abac38d1b5b12a1ccf4f4-20″</span><span style=”font-weight: 400;”>,</span>

<span style=”font-weight: 400;”>           </span><span style=”font-weight: 400;”>”StorageClass”</span><span style=”font-weight: 400;”>: </span><span style=”font-weight: 400;”>”STANDARD”</span><span style=”font-weight: 400;”>,</span>

<span style=”font-weight: 400;”>           </span><span style=”font-weight: 400;”>”Key”</span><span style=”font-weight: 400;”>: </span><span style=”font-weight: 400;”>”/nyctaxi/fbS3nyctaxi/2023_08_08_1691495559589_2.avro”</span><span style=”font-weight: 400;”>,</span>

<span style=”font-weight: 400;”>           </span><span style=”font-weight: 400;”>”Size”</span><span style=”font-weight: 400;”>: </span><span style=”font-weight: 400;”>209748737</span>

<span style=”font-weight: 400;”>       },</span>

<span

Read the rest of this post, which was originally published on Pure Storage.

Previous Post

4 Emerging Technologies for CIOs to Watch in 2023

Next Post

Trino S3 via hive-metastore Integration