final class FileOperation extends Logging

Provides methods for working on files in a stage.

To access an object of this class, use Session.file .

For example: 

// Upload a file to a stage.
session.file.put("file:///tmp/file1.csv", "@myStage/prefix1")
// Download a file from a stage.
session.file.get("@myStage/prefix1/file1.csv", "file:///tmp")
Since

0.4.0

Linear Supertypes
Logging , AnyRef , Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. FileOperation
  2. Logging
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new FileOperation ( session: Session )

Value Members

  1. final def != ( arg0: Any ) : Boolean
    Definition Classes
    AnyRef → Any
  2. final def ## () : Int
    Definition Classes
    AnyRef → Any
  3. final def == ( arg0: Any ) : Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf [ T0 ] : T0
    Definition Classes
    Any
  5. def clone () : AnyRef
    Attributes
    protected[ lang ]
    Definition Classes
    AnyRef
    Annotations
    @throws ( ... ) @native () @HotSpotIntrinsicCandidate ()
  6. def downloadStream ( stageLocation: String , decompress: Boolean ) : InputStream

    Download file from the given stage and return an input stream

    Download file from the given stage and return an input stream

    stageLocation

    Full stage path to the file

    decompress

    True if file compressed

    returns

    An InputStream object

    Since

    1.4.0

  7. final def eq ( arg0: AnyRef ) : Boolean
    Definition Classes
    AnyRef
  8. def equals ( arg0: Any ) : Boolean
    Definition Classes
    AnyRef → Any
  9. def get ( stageLocation: String , targetDirectory: String , options: Map [ String , String ] = Map() ) : Array [ GetResult ]

    Downloads the specified files from a path in a stage (specified by stageLocation ) to the local directory specified by targetLocation .

    Downloads the specified files from a path in a stage (specified by stageLocation ) to the local directory specified by targetLocation .

    This method returns the results as an Array of GetResult objects (one for each file). Each object represents the results of downloading a file.

    For example: 

    // Upload files to a stage.
session.file.put("file:///tmp/file_1.csv", "@myStage/prefix2")
session.file.put("file:///tmp/file_2.csv", "@myStage/prefix2")

// Download one file from a stage.
val res1 = session.file.get("@myStage/prefix2/file_1.csv", "file:///tmp/target")
// Download all the files from @myStage/prefix2.
val res2 = session.file.get("@myStage/prefix2", "file:///tmp/target2")
// Download files with names that match a regular expression pattern.
val getOptions = Map("PATTERN" -> s"'.*file_.*.csv.gz'")
val res3 = session.file.get("@myStage/prefix2", "file:///tmp/target3", getOptions)
    stageLocation

    The location (a directory or filename on a stage) from which you want to download the files. The @ prefix is optional.

    targetDirectory

    The path to the local directory where the file(s) should be downloaded. Specify the path in the following format: file://<path_to_file>/<filename> . If targetDirectory does not already exist, the method creates the directory.

    options

    A Map containing the names and values of optional parameters for the GET command.

    returns

    An Array of PutResult objects (one object for each file downloaded).

    Since

    0.4.0

  10. final def getClass () : Class [_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native () @HotSpotIntrinsicCandidate ()
  11. def hashCode () : Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native () @HotSpotIntrinsicCandidate ()
  12. final def isInstanceOf [ T0 ] : Boolean
    Definition Classes
    Any
  13. def log () : Logger
    Attributes
    protected[ internal ]
    Definition Classes
    Logging
  14. def logDebug ( msg: String , throwable: Throwable ) : Unit
    Attributes
    protected[ internal ]
    Definition Classes
    Logging
  15. def logDebug ( msg: String ) : Unit
    Attributes
    protected[ internal ]
    Definition Classes
    Logging
  16. def logError ( msg: String , throwable: Throwable ) : Unit
    Attributes
    protected[ internal ]
    Definition Classes
    Logging
  17. def logError ( msg: String ) : Unit
    Attributes
    protected[ internal ]
    Definition Classes
    Logging
  18. def logInfo ( msg: String , throwable: Throwable ) : Unit
    Attributes
    protected[ internal ]
    Definition Classes
    Logging
  19. def logInfo ( msg: String ) : Unit
    Attributes
    protected[ internal ]
    Definition Classes
    Logging
  20. def logTrace ( msg: String , throwable: Throwable ) : Unit
    Attributes
    protected[ internal ]
    Definition Classes
    Logging
  21. def logTrace ( msg: String ) : Unit
    Attributes
    protected[ internal ]
    Definition Classes
    Logging
  22. def logWarning ( msg: String , throwable: Throwable ) : Unit
    Attributes
    protected[ internal ]
    Definition Classes
    Logging
  23. def logWarning ( msg: String ) : Unit
    Attributes
    protected[ internal ]
    Definition Classes
    Logging
  24. final def ne ( arg0: AnyRef ) : Boolean
    Definition Classes
    AnyRef
  25. final def notify () : Unit
    Definition Classes
    AnyRef
    Annotations
    @native () @HotSpotIntrinsicCandidate ()
  26. final def notifyAll () : Unit
    Definition Classes
    AnyRef
    Annotations
    @native () @HotSpotIntrinsicCandidate ()
  27. def put ( localFileName: String , stageLocation: String , options: Map [ String , String ] = Map() ) : Array [ PutResult ]

    Uploads the local files specified by localFileName to the stage location specified in stageLocation .

    Uploads the local files specified by localFileName to the stage location specified in stageLocation .

    This method returns the results as an Array of PutResult objects (one for each file). Each object represents the results of uploading a file.

    For example: 

    // Upload a file to a stage without compressing the file.
val putOptions = Map("AUTO_COMPRESS" -> "FALSE")
val res1 = session.file.put("file:///tmp/file1.csv", "@myStage", putOptions)

// Upload the CSV files in /tmp with names that start with "file".
// You can use the wildcard characters "*" and "?" to match multiple files.
val res2 = session.file.put("file:///tmp/file*.csv", "@myStage/prefix2")
    localFileName

    The path to the local file(s) to upload. Specify the path in the following format: file://<path_to_file>/<filename> . (The file:// prefix is optional.) To match multiple files in the path, you can specify the wildcard characters * and ? .

    stageLocation

    The stage (and prefix) where you want to upload the file(s). The @ prefix is optional.

    options

    A Map containing the names and values of optional parameters for the PUT command.

    returns

    An Array of PutResult objects (one object for each file uploaded).

    Since

    0.4.0

  28. final def synchronized [ T0 ] ( arg0: ⇒ T0 ) : T0
    Definition Classes
    AnyRef
  29. def toString () : String
    Definition Classes
    AnyRef → Any
  30. def uploadStream ( stageLocation: String , inputStream: InputStream , compress: Boolean ) : Unit

    Method to compress data from a stream and upload it at a stage location.

    Method to compress data from a stream and upload it at a stage location. The data will be uploaded as one file. No splitting is done in this method.

    caller is responsible for releasing the inputStream after the method is called.

    stageLocation

    Full stage path to the file

    inputStream

    Input stream from which the data will be uploaded

    compress

    Compress data or not before uploading stream

    Since

    1.4.0

  31. final def wait ( arg0: Long , arg1: Int ) : Unit
    Definition Classes
    AnyRef
    Annotations
    @throws ( ... )
  32. final def wait ( arg0: Long ) : Unit
    Definition Classes
    AnyRef
    Annotations
    @throws ( ... ) @native ()
  33. final def wait () : Unit
    Definition Classes
    AnyRef
    Annotations
    @throws ( ... )

Deprecated Value Members

  1. def finalize () : Unit
    Attributes
    protected[ lang ]
    Definition Classes
    AnyRef
    Annotations
    @throws ( classOf[java.lang.Throwable] ) @Deprecated
    Deprecated

Inherited from Logging

Inherited from AnyRef

Inherited from Any

Ungrouped