modin.pandas.to_datetime¶

snowflake.snowpark.modin.pandas.general.to_datetime(arg: DatetimeScalarOrArrayConvertible | DictConvertible | snowflake.snowpark.modin.pandas.DataFrame | Series, errors: DateTimeErrorChoices = 'raise', dayfirst: bool = False, yearfirst: bool = False, utc: bool = False, format: str | None = None, exact: bool | lib.NoDefault = _NoDefault.no_default, unit: str | None = None, infer_datetime_format: lib.NoDefault | bool = _NoDefault.no_default, origin: Any = 'unix', cache: bool = True) → Series | DatetimeScalar | NaTType | None[source]¶

Convert argument to datetime.

This function converts a scalar, array-like, Series or DataFrame/dict-like to a pandas datetime object.

Parameters:

arg (int, float, str, datetime, list, tuple, 1-d array, Series, DataFrame/dict-like) – The object to convert to a datetime. If a DataFrame is provided, the method expects minimally the following columns: "year", "month", "day".
errors ({'ignore', 'raise', 'coerce'}, default 'raise') –
- If 'raise', then invalid parsing will raise an exception.
- If 'coerce', then invalid parsing will be set as NaT.
- If 'ignore', then invalid parsing will return the input.
dayfirst (bool, default False) –
Specify a date parse order if arg is str or is list-like. If True, parses dates with the day first, e.g. "10/11/12" is parsed as 2012-11-10.

Warning

dayfirst=True is not strict, but will prefer to parse with day first. If a delimited date string cannot be parsed in accordance with the given dayfirst option, e.g. to_datetime(['31-12-2021']), then a warning will be shown.
yearfirst (bool, default False) –
Specify a date parse order if arg is str or is list-like.
- If True parses dates with the year first, e.g. "10/11/12" is parsed as 2010-11-12.
- If both dayfirst and yearfirst are True, yearfirst is preceded (same as dateutil).
Warning

yearfirst=True is not strict, but will prefer to parse with year first.
utc (bool, default None) –
Control timezone-related parsing, localization and conversion.
- If True, the function always returns a timezone-aware UTC-localized Timestamp, Series or DatetimeIndex. To do this, timezone-naive inputs are localized as UTC, while timezone-aware inputs are converted to UTC.
- If False (default), inputs will not be coerced to UTC. Timezone-naive inputs will remain naive, while timezone-aware ones will keep their time offsets. Limitations exist for mixed offsets (typically, daylight savings), see Examples section for details.
See also: pandas general documentation about timezone conversion and localization.
format (str, default None) – The strftime to parse time, e.g. "%d/%m/%Y". Note that "%f" will parse all the way up to nanoseconds. See strftime documentation for more information on choices.
exact (bool, default True) –
Control how format is used:
- If True, require an exact format match.
- If False, allow the format to match anywhere in the target string.
unit (str, default 'ns') – The unit of the arg (D,s,ms,us,ns) denote the unit, which is an integer or float number. This will be based off the origin. Example, with unit='ms' and origin='unix', this would calculate the number of milliseconds to the unix epoch start.
infer_datetime_format (bool, default False) – If True and no format is given, attempt to infer the format of the datetime strings based on the first non-NaN element, and if it can be inferred, switch to a faster method of parsing them. In some cases this can increase the parsing speed by ~5-10x.
origin (scalar, default 'unix') –
Define the reference date. The numeric values would be parsed as number of units (defined by unit) since this reference date.
- If 'unix' (or POSIX) time; origin is set to 1970-01-01.
- If 'julian', unit must be 'D', and origin is set to beginning of Julian Calendar. Julian day number 0 is assigned to the day starting at noon on January 1, 4713 BC.
- If Timestamp convertible, origin is set to Timestamp identified by origin.
cache (bool, default True) – cache parameter is ignored with Snowflake backend, i.e., no caching will be applied

Returns:

If parsing succeeded. Return type depends on input (types in parenthesis correspond to fallback in case of unsuccessful timezone or out-of-range timestamp parsing):

scalar: Timestamp (or datetime.datetime)
array-like: Series with datetime64 dtype containing datetime.datetime (or :class: Series of object dtype containing datetime.datetime)
Series: Series of datetime64 dtype (or :class: Series of object dtype containing datetime.datetime)
DataFrame: Series of datetime64 dtype (or Series of object dtype containing datetime.datetime)

Return type:

datetime

Raises:

ParserError – When parsing a date from string fails.
ValueError – When another datetime conversion error happens. For example when one of ‘year’, ‘month’, day’ columns is missing in a DataFrame, or when a Timezone-aware datetime.datetime is found in an array-like of mixed time offsets, and utc=False.

See also

DataFrame.astype: Cast argument to a specified dtype.
to_timedelta: Convert argument to timedelta.
convert_dtypes: Convert dtypes.

Notes

Many input types are supported, and lead to different output types:

scalars can be int, float, str, datetime object (from stdlib datetime module or numpy). They are converted to Timestamp when possible, otherwise they are converted to datetime.datetime. None/NaN/null scalars are converted to NaT.
array-like can contain int, float, str, datetime objects. They are converted to DatetimeIndex when possible, otherwise they are converted to Index with object dtype, containing datetime.datetime. None/NaN/null entries are converted to NaT in both cases.
Series are converted to Series with datetime64 dtype when possible, otherwise they are converted to Series with object dtype, containing datetime.datetime. None/NaN/null entries are converted to NaT in both cases.
DataFrame/dict-like are converted to Series with datetime64 dtype. For each row a datetime is created from assembling the various dataframe columns. Column keys can be common abbreviations like [‘year’, ‘month’, ‘day’, ‘minute’, ‘second’, ‘ms’, ‘us’, ‘ns’]) or plurals of the same.

The following causes are responsible for datetime.datetime objects being returned (possibly inside an Index or a Series with object dtype) instead of a proper pandas designated type (Timestamp or Series with datetime64 dtype):

when any input element is before Timestamp.min or after Timestamp.max, see timestamp limitations.
when utc=False (default) and the input is an array-like or Series containing mixed naive/aware datetime, or aware with mixed time offsets. Note that this happens in the (quite frequent) situation when the timezone has a daylight savings policy. In that case you may wish to use utc=True.

Examples

Handling various input formats

Assembling a datetime from multiple columns of a DataFrame. The keys can be common abbreviations like [‘year’, ‘month’, ‘day’, ‘minute’, ‘second’, ‘ms’, ‘us’, ‘ns’]) or plurals of the same

>>> df = pd.DataFrame({'year': [2015, 2016],
...                    'month': [2, 3],
...                    'day': [4, 5]})
>>> pd.to_datetime(df)
0   2015-02-04
1   2016-03-05
dtype: datetime64[ns]

Copy

Passing infer_datetime_format=True can often-times speedup a parsing if it’s not an ISO8601 format exactly, but in a regular format.

>>> s = pd.Series(['3/11/2000', '3/12/2000', '3/13/2000'] * 1000)
>>> s.head()
0    3/11/2000
1    3/12/2000
2    3/13/2000
3    3/11/2000
4    3/12/2000
dtype: object

Copy

Using a unix epoch time

>>> pd.to_datetime(1490195805, unit='s')
Timestamp('2017-03-22 15:16:45')
>>> pd.to_datetime(1490195805433502912, unit='ns')
Timestamp('2017-03-22 15:16:45.433502912')

Copy

Warning

For float arg, precision rounding might happen. To prevent unexpected behavior use a fixed-width exact type.

Using a non-unix epoch origin

>>> pd.to_datetime([1, 2, 3], unit='D',
...                origin=pd.Timestamp('1960-01-01'))
0   1960-01-02
1   1960-01-03
2   1960-01-04
dtype: datetime64[ns]

Copy

Non-convertible date/times

If a date does not meet the timestamp limitations, passing errors='ignore' will return the original input instead of raising any exception.

Passing errors='coerce' will force an out-of-bounds date to NaT, in addition to forcing non-dates (or non-parseable dates) to NaT.

>>> pd.to_datetime(['13000101', 'abc'], format='%Y%m%d', errors='coerce')
0   NaT
1   NaT
dtype: datetime64[ns]

Copy

Timezones and time offsets

The default behaviour (utc=False) is as follows:

Timezone-naive inputs are converted to timezone-naive Series:

>>> pd.to_datetime(['2018-10-26 12:00', '2018-10-26 13:00:15'])
0   2018-10-26 12:00:00
1   2018-10-26 13:00:15
dtype: datetime64[ns]

Copy

Timezone-aware inputs with constant time offset are still converted to timezone-naive Series by default.

>>> pd.to_datetime(['2018-10-26 12:00:00 -0500', '2018-10-26 13:00:00 -0500'])
0   2018-10-26 12:00:00
1   2018-10-26 13:00:00
dtype: datetime64[ns]

Copy

Use right format to convert to timezone-aware type (Note that when call Snowpark pandas API to_pandas() the timezone-aware output will always be converted to session timezone):

>>> pd.to_datetime(['2018-10-26 12:00:00 -0500', '2018-10-26 13:00:00 -0500'], format="%Y-%m-%d %H:%M:%S %z")
0   2018-10-26 10:00:00-07:00
1   2018-10-26 11:00:00-07:00
dtype: datetime64[ns, America/Los_Angeles]

Copy

Timezone-aware inputs with mixed time offsets (for example issued from a timezone with daylight savings, such as Europe/Paris):

>>> pd.to_datetime(['2020-10-25 02:00:00 +0200', '2020-10-25 04:00:00 +0100'])
0   2020-10-25 02:00:00
1   2020-10-25 04:00:00
dtype: datetime64[ns]

Copy

>>> pd.to_datetime(['2020-10-25 02:00:00 +0200', '2020-10-25 04:00:00 +0100'], format="%Y-%m-%d %H:%M:%S %z")
0   2020-10-24 17:00:00-07:00
1   2020-10-24 20:00:00-07:00
dtype: datetime64[ns, America/Los_Angeles]

Copy

Setting utc=True makes sure always convert to timezone-aware outputs:

Timezone-naive inputs are localized based on the session timezone

>>> pd.to_datetime(['2018-10-26 12:00', '2018-10-26 13:00'], utc=True)
0   2018-10-26 12:00:00-07:00
1   2018-10-26 13:00:00-07:00
dtype: datetime64[ns, America/Los_Angeles]

Copy

Timezone-aware inputs are converted to session timezone

>>> pd.to_datetime(['2018-10-26 12:00:00 -0530', '2018-10-26 12:00:00 -0500'],
...                utc=True)
0   2018-10-26 10:30:00-07:00
1   2018-10-26 10:00:00-07:00
dtype: datetime64[ns, America/Los_Angeles]

Copy