Back to Top

CloudOp

In this topic

  1. Summary
  2. Discussion
  3. Method Overview
  4. Methods
  5. Code sample

Summary

Contains IO methods that are cloud specific.

Discussion

This class cannot be instantiated directly. An instance of this class is returned by the AIO object's cloud property.

Method Overview

MethodExplanation
clearcache (path)

Clears in-memory directory listings, file properties, and BLOB region cache. Call this method when an external process updates folders or files in the cloud store that were previously accessed.

getbucketname ()

Gets the name of a connected bucket or container.

getendpoint (path)

Gets the cloud endpoint specified in the connection.

getfile (src, dst, {options})

Downloads a file from the cloud.

getmetadata (path, {domain})

Gets the metadata of a file or folder.

getobjectstorepath ()

Gets the root path of the connection.

getproperties (path)

Gets the header properties.

getprovidername ()

Gets the cloud provider name.

getprovideroptions ()

Gets the provider options specified in the connection.

getregion ()

Gets the cloud region specified in the connection.

getsignedurl (path, expiry)

Generates a temporary HTTP presigned URL.

putfile (src, dst, {options})

Uploads a file to cloud.

setmetadata (path, {domain}, {metadata}, {options})

Sets the metadata of a file or folder.

validate ({certificate}, {stat}, {list}, {read}, {modify}, {path})

Tests the cloud connection for various file operations. Available permissions may be granular depending on the server ACL configuration.

Methods

clearcache (path)
ParameterExplanationData Type
path

The relative path from the current working directory or an absolute path (absolute vsi path for cloud).

String

This method is only applicable to cloud stores.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.clearcache(r"aio")
getbucketname ()
Return Value
Data TypeExplanation
String

The name of the bucket or container.

This method is only applicable to cloud stores.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.getbucketname()
getendpoint (path)
ParameterExplanationData Type
path

The relative path from the current working directory or an absolute path (absolute vsi path for cloud).

String
Return Value
Data TypeExplanation
String

The cloud endpoint if available.

This method is only applicable to cloud stores.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.getendpoint()
getfile (src, dst, {options})
ParameterExplanationData Type
src

The relative path from the current working directory or an absolute vsi cloud path.

String
dst

The local, or target, path of the file.

String
options

Specifies the options that will be used.

  • RECURSIVE—Subdirectories and their contents will be recursively copied. This option is only applicable for folder operations.
  • SYNC_STRATEGY—The criteria that will be used for overwriting if the target file exists.
  • NUM_THREADS—The number of threads that will be used for parallel file copying.
  • CHUNK_SIZE—The maximum chunk size (in bytes) that will be used to split large objects when uploading or downloading.

Syntax—RECURSIVE=[YES|NO], SYNC_STRATEGY=[TIMESTAMP|ETAG|OVERWRITE], NUM_THREADS=(integer) CHUNK_SIZE=(integer) [x-amz-*|x-goog-*|x-ms-*=(value)]

Example—{'RECURSIVE':'YES', 'SYNC_STRATEGY':'OVERWRITE', 'NUM_THREADS':'10', 'CHUNK_SIZE':'8'}

(The default value is None)

Dictionary
Return Value
Data TypeExplanation
Boolean

True if the file download is successful; otherwise, False.

This method is only applicable to cloud stores.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.getfile(r'utf8_test.json', r"c:\data\datafile.json")
getmetadata (path, {domain})
ParameterExplanationData Type
path

The relative path from the current working directory or an absolute path (absolute vsi path for cloud).

String
domain

The metadata domain, for example, (https://gdal.org/api/cpl.html#_CPPv418VSIGetFileMetadataPKcPKc12CSLConstList).

(The default value is None)

String
Return Value
Data TypeExplanation
Dictionary

A dictionary of the metadata information.

This method is only applicable to cloud stores.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.getmetadata(r'aio')
getobjectstorepath ()
Return Value
Data TypeExplanation
String

The vsi path to the object store.

This method is only applicable to cloud stores.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.getobjectstorepath()
getproperties (path)
ParameterExplanationData Type
path

The relative path from the current working directory or an absolute path (absolute vsi path for cloud).

String
Return Value
Data TypeExplanation
Dictionary

A dictionary of header properties.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.getproperties(r'aio')
getprovidername ()
Return Value
Data TypeExplanation
String

The cloud provider name.

This method is only applicable to cloud stores.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.getprovidername()
getprovideroptions ()
Return Value
Data TypeExplanation
List

The provider options, if available.

This method is only applicable for cloud stores.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.getprovideroptions()
getregion ()
Return Value
Data TypeExplanation
String

The cloud region, if available.

This method is only applicable to cloud stores.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.getregion()
getsignedurl (path, expiry)
ParameterExplanationData Type
path

The relative path from the current working directory or an absolute path (absolute vsi path for cloud).

String
expiry

The expiry time in seconds.

Integer
Return Value
Data TypeExplanation
String

The HTTP signed URL.

This method is only applicable to cloud stores.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.getsignedurl(r'data.json', 5)
putfile (src, dst, {options})
ParameterExplanationData Type
src

The local (source) path of the file.

String
dst

The relative path from the current working directory or an absolute vsi cloud path.

String
options

Specifies the options that will be used.

  • RECURSIVE—Subdirectories and their contents will be recursively copied. This option is only applicable to folder operations.
  • SYNC_STRATEGY—The criteria that will be used for overwriting if the target file exists.
  • NUM_THREADS—The number of threads that will be used for parallel file copying.
  • CHUNK_SIZE—The maximum chunk size (in bytes) that will be used to split large objects when uploading or downloading.

Syntax—RECURSIVE=[YES|NO], SYNC_STRATEGY=[TIMESTAMP|ETAG|OVERWRITE], NUM_THREADS=(integer) CHUNK_SIZE=(integer) [x-amz-*|x-goog-*|x-ms-*=(value)]

Example—{'RECURSIVE':'YES', 'SYNC_STRATEGY':'OVERWRITE', 'NUM_THREADS':'10', 'CHUNK_SIZE':'8'}

(The default value is None)

Dictionary
Return Value
Data TypeExplanation
Boolean

True if the file upload is successful; otherwise, False.

This method is only applicable for cloud.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.putfile(r"C:\data\data_file.json", r'data_folder\data_file_new.json')
setmetadata (path, {domain}, {metadata}, {options})
ParameterExplanationData Type
path

The relative path from the current working directory or an absolute path (absolute vsi path for cloud).

String
domain

The metadata domain, for example, (https://gdal.org/api/cpl.html#_CPPv418VSISetFileMetadataPKc12CSLConstListPKc12CSLConstList).

(The default value is None)

String
metadata

The metadata that will be set.

(The default value is None)

Dictionary
options

The options that will be used to set the metadata.

(The default value is None)

Dictionary
Return Value
Data TypeExplanation
Boolean

True if the metadata is set successfully; otherwise, False.

This method is only applicable to cloud stores.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.setmetadata(
    r"cog.tif",
    "HEADERS",
    {
        "x-amz-id-2": "HxM4BcALW+HnJ13owxzMpeQXXI59VKK3I9zujbqyLBCinCt6prnzKXR4Ixhb/AloFVJLGtuqKG7nSLCLaxObTw==",
        "x-amz-request-id": "KYTDPC2WH2ZQK2D2",
        "Date": "Tue, 13 Jun 2023 13:39:33 GMT",
        "Last-Modified": "Wed, 16 Sep 2020 13:49:34 GMT",
        "ETag": '"c220c099c46c60703de0c55763d04371"',
        "x-amz-meta-s3b-last-modified": "20171213T210220Z",
        "Content-Disposition": "attachment",
        "Accept-Ranges": "bytes",
        "Content-Range": "bytes 0-16383/4339079",
        "Content-Type": "text/plain",
        "Server": "AmazonS3",
        "Content-Length": "4545",
    },
)
validate ({certificate}, {stat}, {list}, {read}, {modify}, {path})
ParameterExplanationData Type
certificate

Verifies the SSL certificate and Online Certificate Status Protocol (OCSP) certificate check.

Boolean
stat

Tests stat on the root container or connected folder.

Boolean
list

Tests list directory access.

Boolean
read

Tests read file permission

Boolean
modify

Tests the write and delete permissions. If delete access is denied, a test file will remain on the cloud.

Boolean
path

The path to the file that will be used for read test. If not provided, the first entry from the current directory listing is used.

String
Return Value
Data TypeExplanation
List

Results of the test and a detailed error message if available.

This method is only applicable for cloud.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.validate(True, True, True, True)

Feedback on this topic?

In this topic
  1. Summary
  2. Discussion
  3. Method Overview
  4. Methods
  5. Code sample