AIO

サマリー

Creates an AIO object to handle local and cloud files, and data stores in a unified way that can be used in Python.

Using this object, you can perform tasks such as copying files from one location to another and renaming or deleting a folder, as well as filing related operations including read and write. The AIO object can be used for cloud stores by specifying the path to an .acs file as the input. For a local data store, create an AIO object by specifying a local folder path, which will then be used as the current working directory, or by not specifying any parameter values.

説明

The AIO object supports file operations agnostic of the file system. The object currently supports Windows, Linux and major cloud storage implementations.

Apart from a local file system and UNC paths, the AIO object supports the following virtual cloud URIs for class initialization:

• ACS Connection files (.acs)—Stores connection information in binary persisted encrypted format. See the Create Cloud Storage Connection File tool for more information.
• /cloudStores json—Stores connection information in JSON format for ArcGIS Enterprise.
• /vsi paths—GDAL is the preferred URI for file systems in agnostic file operations. See GDAL Virtual File Systems for more information.
• http[s] paths—The cloud provider URL for a resource, which sometimes contains connection information such as Microsoft Azure SAS.
• A cloud provider-specific URI schema prefix such as s3:// or abfs://.

注意:

Connection information can also be used through IAM/RBAC, GDAL environment options, and provider-specific CLI configuration files in the .aws, .azure, or bin/gdalrc folders.

All cloud-specific methods can be accessed using the cloud property.

When working with cloud operations, class methods support relative paths to the current working directory (aio.getcwd). If absolute paths are used, only vsi paths are supported. These types of operations return vsi paths by default. The type of output path can be overridden by providing the CloudPathType enum.

These methods follow the same conventions and behavior as Python native file system methods.

構文

AIO ({path})

パラメーター	説明	データ タイプ
path	To construct an AIO object to handle a cloud store, use the .acs file path. To construct an AIO object to handle a local file store, this parameter can be a local folder path or it can be left empty. If a local folder path is provided, it will be used as the current working directory.	String

プロパティ

プロパティ	説明	データ タイプ
cloud (読み取り専用)	An instance of a class that has cloud-specific methods. It is used to retrieve details of the cloud store that was used when the AIO object was created. Returns a CloudOp object. cloud_io = AIO(r"C:\data\datacloud.acs") cloud_io.cloud.getprovidername()	Object

方法の概要

方法	説明
chdir (path)	Changes the working directory relative to the current working directory..
close (handle)	Closes the open file handle.
copy (src, dst, {options})	Copies a file from the source to the destination. For a cloud-based AIO object: If the src or dst value is a local path, use an absolute local path. If the src or dst value is a cloud path, use a relative path from the current working directory or an absolute vsi cloud path. If the src and dst values are cloud paths, use absolute vsi cloud paths. For a local AIO object: The src and dst values can be a relative path from the current working directory or an absolute path.
copytree (src, dst, {options})	Copies the contents from one directory to another. For a cloud-based AIO object: If the src or dst value is a local path, use an absolute local path. If the src or dst value is a cloud path, use a relative path from the current working directory or an absolute vsi cloud path. If the src and dst values are cloud paths, use absolute vsi cloud paths. For a local AIO object: The src and dst values can be a relative path from the current working directory or an absolute path.
exists (path)	Verifies whether a path exists.
getatime (path)	Gets the file access time for a file.
getctime (path)	Gets the file creation time for a file.
getcwd ({type})	Gets the current working directory.
getmtime (path)	Gets the file modification time for a file.
getpath (path, {type})	Constructs a URI path.
getsize (path)	Gets the size of a file in bytes.
isdir (path)	Verifies whether a path is a directory.
isfile (path)	Verifies whether a path is a file.
listdir ({path}, {type})	Lists the contents of a directory nonrecursively.
makedirs (path)	Creates a directory recursively.
mkdir (path)	Creates a directory.
open (path, {mode}, {encoding}, {mime})	Opens a file handle. The handle must be explicitly closed.
remove (path)	Deletes a file.
removefiles (path)	Delete multiple files.
rename (src, dst)	Renames or moves a file or a directory. Renaming cloud files is computationally intensive, as copy and delete are performed on the server side. For a cloud based AIO object: If the src or dst value is a local path, use an absolute local path. If the src or dst value is a cloud path, use a relative path from the current working directory or an absolute vsi cloud path. If the src and dst values are cloud paths, use absolute vsi cloud paths. For a local AIO object: The src and dst values can be a relative path from the current working directory or an absolute path.
rmtree (path)	Removes a directory and its contents recursively.
scandir (path, {depth}, {type})	Returns an iterable of entries in a directory.

方法

chdir (path)

パラメーター	説明	データ タイプ
path	A relative path from the current working directory or an absolute path (absolute vsi path for cloud).	String

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.chdir(r"aio")  # relative path

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.chdir(r"/vsis3/u-agu/arcio")  # absolute VSI path

local_io = AIO(r"C:\data")
local_io.chdir(r"aio")

close (handle)

パラメーター	説明	データ タイプ
handle	An open file handle. The data type is AIOFile. 注意: An instance of this class is returned by the AIO object's open method.	Object

cloud_io = AIO(r"C:\data\datacloud.acs")
rcsfile = cloud_io.open(r'testfile.txt', 'r')
rcsfile.read(size=4)
cloud_io.close(rcsfile)

copy (src, dst, {options})

パラメーター	説明	データ タイプ
src	The source path of the file.	String
dst	The target path of the file.	String
options	Specifies the options that will be used. RECURSIVE—Subdirectories and their contents will be recursively copied. This option is only applicable for folder operations. SYNC_STRATEGY—The criteria that will be used for overwriting if the target file exists. NUM_THREADS—The number of threads that will be used for parallel file copying. CHUNK_SIZE—The maximum chunk size (in bytes) that will be used to split large objects when uploading or downloading. Syntax—RECURSIVE=[YES|NO], SYNC_STRATEGY=[TIMESTAMP|ETAG|OVERWRITE], NUM_THREADS=(integer) CHUNK_SIZE=(integer) [x-amz-*|x-goog-*|x-ms-*=(value)] Example—{'RECURSIVE':'YES', 'SYNC_STRATEGY':'OVERWRITE', 'NUM_THREADS':'10', 'CHUNK_SIZE':'8'} (デフォルト値は次のとおりです None)	Dictionary

戻り値

データ タイプ	説明
String	The destination path.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.copy(r'/vsis3/data/list/utf8_test.json', 
              r'/vsis3/data/list/new_dir_new/utf8_test.json')

local_io = AIO(r"C:\data")
local_io.copy(r"C:\data\datatest.json", r"C:\data_temp\datatest.json")

copytree (src, dst, {options})

パラメーター	説明	データ タイプ
src	The source path of the file.	String
dst	The target path of the file.	String
options	Specifies the options that will be used. RECURSIVE—Subdirectories and their contents will be recursively copied. This option is only applicable for folder operations. SYNC_STRATEGY—The criteria that will be used for overwriting if the target file exists. NUM_THREADS—The number of threads that will be used for parallel file copying. CHUNK_SIZE—The maximum chunk size (in bytes) that will be used to split large objects when uploading or downloading. Syntax—RECURSIVE=[YES|NO], SYNC_STRATEGY=[TIMESTAMP|ETAG|OVERWRITE], NUM_THREADS=(integer) CHUNK_SIZE=(integer) [x-amz-*|x-goog-*|x-ms-*=(value)] Example— {'RECURSIVE':'YES', 'SYNC_STRATEGY':'OVERWRITE', 'NUM_THREADS':'10', 'CHUNK_SIZE':'8'} (デフォルト値は次のとおりです None)	Dictionary

戻り値

データ タイプ	説明
String	The destination path.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.copytree(r'/vsis3/data/list', r'/vsis3/data/data_new_dir')

local_io = AIO(r"C:\data")
local_io.copytree(r"C:\data\list", r"C:\data\data_new_dir")

exists (path)

パラメーター	説明	データ タイプ
path	A relative path from the current working directory or an absolute path (absolute vsi path for cloud).	String

戻り値

データ タイプ	説明
Boolean	True if the directory or file exists; otherwise, False.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.exists(r"C:\data\datacloud.acs\data")

getatime (path)

パラメーター	説明	データ タイプ
path	The path for which the access time will be obtained. The value can be a relative path from the current working directory or an absolute path.	String

戻り値

データ タイプ	説明
Double	The access time, in number of seconds since epoch.

This method is not available for cloud storage.

local_io = AIO(r"C:\data")
local_io.getatime(r"data.json")

getctime (path)

パラメーター	説明	データ タイプ
path	The path for which the creation time will be obtained. The value can be a relative path from the current working directory or an absolute path.	String

戻り値

データ タイプ	説明
Double	The creation time, in number of seconds since epoch.

This method is not available for cloud storage.

local_io = AIO(r"C:\data")
local_io.getctime(r"data.json")

getcwd ({type})

パラメーター	説明	データ タイプ
type	Specifies the path style for the returned URI (for cloud only). The data type is CloudPathType. Available options are CloudPathType.VSI, CloudPathType.ACS, CloudPathType.HTTP, and CloudPathType.CLOUDSTORES. (デフォルト値は次のとおりです CloudPathType.VSI)	Object

戻り値

データ タイプ	説明
String	The current working directory.

local_io = AIO()
local_io.getcwd()

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.getcwd()

getmtime (path)

パラメーター	説明	データ タイプ
path	The relative path from the current working directory or an absolute path (absolute vsi path for cloud).	String

戻り値

データ タイプ	説明
Double	The file modification time in number of seconds since epoch.

local_io = AIO(r"C:\data")
local_io.getmtime(r"data.json")

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.getmtime(r"data.json")

getpath (path, {type})

パラメーター	説明	データ タイプ
path	The relative path from the current working directory or an absolute path (absolute vsi path for cloud).	String
type	Specifies the path style for the returned URI (for cloud only). The data type is CloudPathType. Available options are CloudPathType.VSI, CloudPathType.ACS, CloudPathType.HTTP, and CloudPathType.CLOUDSTORES. (デフォルト値は次のとおりです CloudPathType.VSI)	Object

戻り値

データ タイプ	説明
String	The constructed URI path.

from arcpy import CloudPathType
cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.getpath('cog.tif', CloudPathType.HTTP)

getsize (path)

パラメーター	説明	データ タイプ
path	The relative path from the current working directory or an absolute path (absolute vsi path for cloud).	String

戻り値

データ タイプ	説明
Integer	The size of the file in bytes.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.getsize(r'cog.tif')

isdir (path)

パラメーター	説明	データ タイプ
path	A relative path from the current working directory or an absolute path (absolute vsi path for cloud).	String

戻り値

データ タイプ	説明
Boolean	True if the directory exists; otherwise, False.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.isdir(r'cog')

isfile (path)

パラメーター	説明	データ タイプ
path	A relative path from the current working directory or an absolute path (absolute vsi path for cloud).	String

戻り値

データ タイプ	説明
Boolean	True if the file exists; otherwise, False.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.isfile(r'cog')

listdir ({path}, {type})

パラメーター	説明	データ タイプ
path	The relative path from the current working directory or an absolute path (absolute vsi path for cloud). (デフォルト値は次のとおりです None)	String
type	Specifies the path style for the returned URI (for cloud only). The data type is CloudPathType. Available options are CloudPathType.VSI, CloudPathType.ACS, CloudPathType.HTTP, and CloudPathType.CLOUDSTORES. (デフォルト値は次のとおりです CloudPathType.VSI)	Object

戻り値

データ タイプ	説明
String	A list of file names in the folder.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.listdir('data_folder')

from arcpy import CloudPathType
cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.listdir('data_folder', CloudPathType.ACS)

local_io = AIO(r"C:\data")
local_io.listdir()

makedirs (path)

パラメーター	説明	データ タイプ
path	A relative path from the current working directory or an absolute path (absolute vsi path for cloud).	String

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.makedirs(r"datasensor\datatest\data")

mkdir (path)

パラメーター	説明	データ タイプ
path	A relative path from the current working directory or an absolute path (absolute vsi path for cloud).	String

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.mkdir(r'cog')

open (path, {mode}, {encoding}, {mime})

パラメーター	説明	データ タイプ
path	A relative path from the current working directory or an absolute path (absolute vsi path for cloud).	String
mode	Specifies the mode that will be used for opening a file. r—The file will be opened in read mode. w—The file will be opened in write mode. rb—The file will be opened in binary format for reading. wb—The file will be opened in binary format for writing. 注意: Only read mode honors the encoding parameter. (デフォルト値は次のとおりです None)	String
encoding	The type of encoding that will be used to open a file. utf-8—UTF-8 encoding will be used. This is the default. utf-16—UTF-16 encoding will be used. ascii—ASCII encoding will be used. (デフォルト値は次のとおりです utf-8)	String
mime	The Multipurpose Internet Mail Extensions (MIME) headers. It is recommended that you set MIME headers when creating the file for faster creation and setting of metadata. (デフォルト値は次のとおりです None)	Dictionary

戻り値

データ タイプ	説明
File	Returns the AIOFile object.

from arcpy import AIO
local_io = AIO(r"C:\data")
rcsfile = local_io.open(r'testfile.txt', 'r')
rcsfile.write("This is a test file.")
rcsfile.close()

from arcpy import AIO
cloud_io = AIO(r"C:\data\datacloud.acs")
rcsfile = cloud_io.open(r"C:\data\info\datafile.txt", 'w', mime={'Content-Type': 'text/plain'})
rcsfile.write("This is a test file.")
rcsfile.close()

# Using with statement (context manager to close the opened file)
cloud_io = AIO(r"C:\data\datacloud.acs")
with cloud_io.open(r"C:\data\info\datafile.txt", 'w', mime={'Content-Type': 'text/plain'}) as rcsfile:
    rcsfile.write("This is a test file.")

remove (path)

パラメーター	説明	データ タイプ
path	A relative path from the current working directory or an absolute path (absolute vsi path for cloud).	String

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.remove(r'rcsfile.txt')

local_io = AIO(r"c:\data")
local_io.remove(r'rcsfile.txt')

removefiles (path)

パラメーター	説明	データ タイプ
path [path,...]	The relative path from the current working directory or an absolute path (absolute vsi path for cloud).	String

戻り値

データ タイプ	説明
List	Returns the list of paths that were not removed in case of failure.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.removefiles(['/vsis3/data/list/to_be_deleted/datatest.json','/vsis3/u-agu/list/to_be_deleted/data_new.json'])

local_io = AIO(r"c:\data")
local_io.removefiles([r'C:\data\datatest.json',r'C:\data\data_new.json'])

rename (src, dst)

パラメーター	説明	データ タイプ
src	The source path of the file.	String
dst	The target path of the file.	String

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.rename(r'to_be_renamed/cog.tif',r'to_be_renamed/cognew.tif')

local_io = AIO(r"C:\data")
local_io.rename(r'test.json',r'new_test.json')

rmtree (path)

パラメーター	説明	データ タイプ
path	The relative path from the current working directory or an absolute path (absolute vsi path for cloud).	String

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.rmtree(r"aio")  # Relative path

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.rmtree(r"/vsis3/data/arcio")  # Absolute VSI path

local_io = AIO(r"C:\data")
local_io.rmtree(r"aio")

scandir (path, {depth}, {type})

パラメーター	説明	データ タイプ
path	The relative path from the current working directory or an absolute path (absolute vsi path for cloud).	String
depth	The recursion depth. A value of -1 is for recursive scanning. A value of 0 is for scanning the current directory. A value greater than 0 (>0) is for an arbitrary level. (デフォルト値は次のとおりです 0)	Integer
type	Specifies the path style for the returned URI (for cloud only). The data type is CloudPathType. Available options are CloudPathType.VSI, CloudPathType.ACS, CloudPathType.HTTP, and CloudPathType.CLOUDSTORES. (デフォルト値は次のとおりです CloudPathType.VSI)	Object

戻り値

データ タイプ	説明
Iterable	An iterable of the AIODirEntry object.

cloud_io = AIO(r"C:\data\datacloud.acs")
# Depth = 0, performs cur dir scanning only
from arcpy import CloudPathType
for item in cloud_io.scandir(r'list', depth=0, type=CloudPathType.HTTP):
    print(item.path)
    print(item.is_dir())
    print(item.is_file())
    ob = item.stat()
    print(ob.st_mode)
    # Cloud specific operations through cloud property
    print(item.cloud.getvsipath())
    print(item.cloud.getpath(CloudPathType.ACS))

local_io = AIO(r"c:\data")
# Depth = -1, performs recursive scanning
for item in local_io.scandir(r'aio', -1):
    print(item.name)
    print(item.path)
    print(item.is_dir())
    print(item.is_file())
    ob = item.stat()
    print(ob.st_mode)

コードのサンプル

# To create an AIO object based on cloud store
cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.getcwd()
cloud_io.listdir(r'aio') 
# aio is a virtual folder in the cloud store referred in datacloud.acs

# To create an AIO object based on local file system
local_io = AIO()
local_io.getcwd()  # Returns the current working directory

# To create an AIO object based on local file system with path parameter
local_io = AIO(r"C:\data")  # Set the current working directory as C:\data
local_io.getcwd()  # Return the current working directory that is C:\data