Snowpark Migration Accelerator : Notes de version¶

Modifications¶

• Updated .NET version to v10.0.0.

• Bumped Python AST and Parser version to v149.1.23.

• The SMA now correctly identifies and reports usages of org.apache.spark.sql.functions.to_number and org.apache.spark.sql.functions.try_to_number as unsupported elements within the Snowpark API.

Ajouté¶

• Output folder validation: The output folder selection now validates for existing project files (.snowct). If the selected folder already contains a project, the selection is blocked and an error is displayed, preventing accidental overwrites.

• Dual-assessment workflow: Automatically evaluates both SCOS and Snowpark API conversion paths, recommending the best fit for the user’s workload.

• Two conversion modes: Snowpark Connect (SCOS) and Snowpark API are now available as distinct conversion targets.

• Readiness score section: Displays the recommended conversion target with a color-coded compatibility score badge.

• File compatibility breakdown: KPI cards showing fully compatible files, files requiring changes, and files with unsupported APIs.

• Data distribution charts: Stacked bar charts showing input data sources and output data targets, grouped by platform.

• Code dependencies: Interactive donut chart categorizing dependencies as Supported, Internal, or Unknown.

• Issues by category: AI-enriched table grouping conversion issues into human-readable categories with file counts and key issue summaries. Displayed when a Snowflake session is active.

• Execution summary: Project metadata, engine version information, and input/output folder references.

• Performance: Assessment report data is cached for the duration of the session, enabling instant page loads when navigating back to results.

Modifications¶

• Card layout improvements: Assessment and conversion cards were refactored for improving the user experience.

• Assessment workflow shortcut: If an assessment has already been completed, clicking « Analyze code » navigates directly to the results page instead of re-running the assessment.

• Renamed connection action: « Activate Assistant » is now « Connect to Snowflake » across the application header and connection dialog for clearer terminology.

Included SMA Core Version¶

• Snowpark Conversion Core: 8.1.60

Included SnowConvert AI Version¶

• SnowConvert AI Version 2.2.0 (Release Notes)

Ajouté¶

• Added support for processing files located in a hidden folder (such as .databricks when exported from the source). These files are now correctly processed by the SMA.

• Added 245 new PySpark elements to the SMA mapping table with a NotSupported status. These entries correspond to functions and methods introduced in PySpark 3.3.0 through 4.1.x:

  • 219 functions (pyspark.sql.functions)

  • 4 DataFrame methods

  • 3 Column methods

  • 5 Session methods

  • 2 ReadWriter methods

  • 12 Types classes

• Added new EWIs for the following Pandas elements:

  • PNDSPY1019: pandas.core.arrays.datetimelike.DatelikeOps.strftime partial support

  • PNDSPY1020: pandas.core.arrays.datetimelike.TimelikeOps.ceil partial support

  • PNDSPY1021: pandas.core.arrays.datetimelike.TimelikeOps.floor partial support

  • PNDSPY1022: pandas.core.arrays.datetimelike.TimelikeOps.round partial support

  • PNDSPY1023: pandas.core.arrays.datetimes.DatetimeArray.day_name partial support

  • PNDSPY1024: pandas.core.arrays.datetimes.DatetimeArray.month_name partial support

  • PNDSPY1025: pandas.core.arrays.datetimes.DatetimeArray.tz_convert partial support

  • PNDSPY1026: pandas.core.arrays.datetimes.DatetimeArray.tz_localize partial support

  • PNDSPY1027: pandas.core.base.IndexOpsMixin.argmax partial support

  • PNDSPY1028: pandas.core.base.IndexOpsMixin.argmin partial support

  • PNDSPY1029: pandas.core.base.IndexOpsMixin.value_counts partial support

  • PNDSPY1030: pandas.core.frame.DataFrame.T partial support

  • PNDSPY1031: pandas.core.frame.DataFrame._*dataframe*_ partial support

  • PNDSPY1032: pandas.core.frame.DataFrame.add partial support

  • PNDSPY1033: pandas.core.frame.DataFrame.align partial support

  • PNDSPY1034: pandas.core.frame.DataFrame.all partial support

  • PNDSPY1035: pandas.core.frame.DataFrame.any partial support

  • PNDSPY1036: pandas.core.frame.DataFrame.applymap partial support

  • PNDSPY1037: pandas.core.frame.DataFrame.asfreq partial support

  • PNDSPY1038: pandas.core.frame.DataFrame.astype partial support

  • PNDSPY1039: pandas.core.frame.DataFrame.at partial support

  • PNDSPY1040: pandas.core.frame.DataFrame.backfill partial support

  • PNDSPY1041: pandas.core.frame.DataFrame.bfill partial support

  • PNDSPY1042: pandas.core.frame.DataFrame.compare partial support

  • PNDSPY1043: pandas.core.frame.DataFrame.corr partial support

  • PNDSPY1044: pandas.core.frame.DataFrame.cumsum partial support

  • PNDSPY1045: pandas.core.frame.DataFrame.div partial support

  • PNDSPY1046: pandas.core.frame.DataFrame.divide partial support

  • PNDSPY1047: pandas.core.frame.DataFrame.dropna partial support

  • PNDSPY1048: pandas.core.frame.DataFrame.eq partial support

  • PNDSPY1049: pandas.core.frame.DataFrame.eval partial support

  • PNDSPY1050: pandas.core.frame.DataFrame.expanding partial support

  • PNDSPY1051: pandas.core.frame.DataFrame.ffill partial support

  • PNDSPY1052: pandas.core.frame.DataFrame.fillna partial support

  • PNDSPY1053: pandas.core.frame.DataFrame.floordiv partial support

  • PNDSPY1054: pandas.core.frame.DataFrame.from_records partial support

  • PNDSPY1055: pandas.core.frame.DataFrame.ge partial support

  • PNDSPY1056: pandas.core.frame.DataFrame.groupby partial support

  • PNDSPY1057: pandas.core.frame.DataFrame.gt partial support

  • PNDSPY1058: pandas.core.frame.DataFrame.idxmax partial support

  • PNDSPY1059: pandas.core.frame.DataFrame.idxmin partial support

  • PNDSPY1060: pandas.core.frame.DataFrame.info partial support

  • PNDSPY1061: pandas.core.frame.DataFrame.join partial support

  • PNDSPY1062: pandas.core.frame.DataFrame.le partial support

  • PNDSPY1063: pandas.core.frame.DataFrame.loc partial support

  • PNDSPY1064: pandas.core.frame.DataFrame.lt partial support

  • PNDSPY1065: pandas.core.frame.DataFrame.map partial support

  • PNDSPY1066: pandas.core.frame.DataFrame.mask partial support

  • PNDSPY1067: pandas.core.frame.DataFrame.melt partial support

  • PNDSPY1068: pandas.core.frame.DataFrame.merge partial support

  • PNDSPY1069: pandas.core.frame.DataFrame.mod partial support

  • PNDSPY1070: pandas.core.frame.DataFrame.mul partial support

  • PNDSPY1071: pandas.core.frame.DataFrame.multiply partial support

  • PNDSPY1072: pandas.core.frame.DataFrame.ne partial support

  • PNDSPY1073: pandas.core.frame.DataFrame.nlargest partial support

  • PNDSPY1074: pandas.core.frame.DataFrame.nsmallest partial support

  • PNDSPY1075: pandas.core.frame.DataFrame.nunique partial support

  • PNDSPY1076: pandas.core.frame.DataFrame.pad partial support

  • PNDSPY1077: pandas.core.frame.DataFrame.pct_change partial support

  • PNDSPY1078: pandas.core.frame.DataFrame.pivot partial support

  • PNDSPY1079: pandas.core.frame.DataFrame.pivot_table partial support

  • PNDSPY1080: pandas.core.frame.DataFrame.pow partial support

  • PNDSPY1081: pandas.core.frame.DataFrame.quantile partial support

  • PNDSPY1082: pandas.core.frame.DataFrame.radd partial support

  • PNDSPY1083: pandas.core.frame.DataFrame.rank partial support

  • PNDSPY1084: pandas.core.frame.DataFrame.rdiv partial support

  • PNDSPY1085: pandas.core.frame.DataFrame.reindex partial support

  • PNDSPY1086: pandas.core.frame.DataFrame.rename partial support

  • PNDSPY1087: pandas.core.frame.DataFrame.replace partial support

  • PNDSPY1088: pandas.core.frame.DataFrame.resample partial support

  • PNDSPY1089: pandas.core.frame.DataFrame.rfloordiv partial support

  • PNDSPY1090: pandas.core.frame.DataFrame.rmod partial support

  • PNDSPY1091: pandas.core.frame.DataFrame.rmul partial support

  • PNDSPY1092: pandas.core.frame.DataFrame.rolling partial support

  • PNDSPY1093: pandas.core.frame.DataFrame.round partial support

  • PNDSPY1094: pandas.core.frame.DataFrame.rpow partial support

  • PNDSPY1095: pandas.core.frame.DataFrame.rsub partial support

  • PNDSPY1096: pandas.core.frame.DataFrame.rtruediv partial support

  • PNDSPY1097: pandas.core.frame.DataFrame.sample partial support

  • PNDSPY1098: pandas.core.frame.DataFrame.shift partial support

  • PNDSPY1099: pandas.core.frame.DataFrame.skew partial support

  • PNDSPY1100: pandas.core.frame.DataFrame.sort_index partial support

  • PNDSPY1101: pandas.core.frame.DataFrame.sort_values partial support

  • PNDSPY1102: pandas.core.frame.DataFrame.stack partial support

  • PNDSPY1103: pandas.core.frame.DataFrame.std partial support

  • PNDSPY1104: pandas.core.frame.DataFrame.sub partial support

  • PNDSPY1105: pandas.core.frame.DataFrame.subtract partial support

  • PNDSPY1106: pandas.core.frame.DataFrame.to_csv partial support

  • PNDSPY1107: pandas.core.frame.DataFrame.transform partial support

  • PNDSPY1108: pandas.core.frame.DataFrame.transpose partial support

  • PNDSPY1109: pandas.core.frame.DataFrame.truediv partial support

  • PNDSPY1110: pandas.core.frame.DataFrame.tz_convert partial support

  • PNDSPY1111: pandas.core.frame.DataFrame.tz_localize partial support

  • PNDSPY1112: pandas.core.frame.DataFrame.unstack partial support

  • PNDSPY1113: pandas.core.frame.DataFrame.var partial support

  • PNDSPY1114: pandas.core.frame.DataFrame.where partial support

  • PNDSPY1115: pandas.core.generic.NDFrame.shift partial support

  • PNDSPY1116: pandas.core.groupby.generic.DataFrameGroupBy.agg partial support

  • PNDSPY1117: pandas.core.groupby.generic.DataFrameGroupBy.aggregate partial support

  • PNDSPY1118: pandas.core.groupby.generic.DataFrameGroupBy.fillna partial support

  • PNDSPY1119: pandas.core.groupby.generic.DataFrameGroupBy.idxmax partial support

  • PNDSPY1120: pandas.core.groupby.generic.DataFrameGroupBy.idxmin partial support

  • PNDSPY1121: pandas.core.groupby.generic.DataFrameGroupBy.transform partial support

  • PNDSPY1122: pandas.core.groupby.generic.DataFrameGroupBy.value_counts partial support

  • PNDSPY1123: pandas.core.groupby.groupby.BaseGroupBy.get_group partial support

  • PNDSPY1124: pandas.core.groupby.groupby.GroupBy.all partial support

  • PNDSPY1125: pandas.core.groupby.groupby.GroupBy.any partial support

  • PNDSPY1126: pandas.core.groupby.groupby.GroupBy.apply partial support

  • PNDSPY1127: pandas.core.groupby.groupby.GroupBy.bfill partial support

  • PNDSPY1128: pandas.core.groupby.groupby.GroupBy.ffill partial support

  • PNDSPY1129: pandas.core.groupby.groupby.GroupBy.first partial support

  • PNDSPY1130: pandas.core.groupby.groupby.GroupBy.last partial support

  • PNDSPY1131: pandas.core.groupby.groupby.GroupBy.pct_change partial support

  • PNDSPY1132: pandas.core.groupby.groupby.GroupBy.quantile partial support

  • PNDSPY1133: pandas.core.groupby.groupby.GroupBy.resample partial support

  • PNDSPY1134: pandas.core.groupby.groupby.GroupBy.rolling partial support

  • PNDSPY1135: pandas.core.groupby.groupby.GroupBy.shift partial support

  • PNDSPY1136: pandas.core.groupby.groupby.GroupBy.std partial support

  • PNDSPY1137: pandas.core.groupby.groupby.GroupBy.var partial support

  • PNDSPY1138: pandas.core.indexes.base.Index.all partial support

  • PNDSPY1139: pandas.core.indexes.base.Index.any partial support

  • PNDSPY1140: pandas.core.indexes.base.Index.nlevels partial support

  • PNDSPY1141: pandas.core.indexes.base.Index.reindex partial support

  • PNDSPY1142: pandas.core.indexes.base.Index.sort_values partial support

  • PNDSPY1143: pandas.core.indexes.datetimes.DatetimeIndex.ceil partial support

  • PNDSPY1144: pandas.core.indexes.datetimes.DatetimeIndex.day_name partial support

  • PNDSPY1145: pandas.core.indexes.datetimes.DatetimeIndex.floor partial support

  • PNDSPY1146: pandas.core.indexes.datetimes.DatetimeIndex.month_name partial support

  • PNDSPY1147: pandas.core.indexes.datetimes.DatetimeIndex.round partial support

  • PNDSPY1148: pandas.core.indexes.datetimes.DatetimeIndex.std partial support

  • PNDSPY1149: pandas.core.indexes.datetimes.DatetimeIndex.tz_convert partial support

  • PNDSPY1150: pandas.core.indexes.datetimes.DatetimeIndex.tz_localize partial support

  • PNDSPY1151: pandas.core.indexes.datetimes.bdate_range partial support

  • PNDSPY1152: pandas.core.indexes.datetimes.date_range partial support

  • PNDSPY1153: pandas.core.resample.Resampler.asfreq partial support

  • PNDSPY1154: pandas.core.resample.Resampler.bfill partial support

  • PNDSPY1155: pandas.core.resample.Resampler.ffill partial support

  • PNDSPY1156: pandas.core.resample.Resampler.fillna partial support

  • PNDSPY1157: pandas.core.resample.Resampler.first partial support

  • PNDSPY1158: pandas.core.resample.Resampler.last partial support

  • PNDSPY1159: pandas.core.resample.Resampler.quantile partial support

  • PNDSPY1160: pandas.core.resample.Resampler.std partial support

  • PNDSPY1161: pandas.core.resample.Resampler.var partial support

  • PNDSPY1162: pandas.core.reshape.concat.concat partial support

  • PNDSPY1163: pandas.core.reshape.melt.melt partial support

  • PNDSPY1164: pandas.core.reshape.merge.merge partial support

  • PNDSPY1165: pandas.core.reshape.merge.merge_asof partial support

  • PNDSPY1166: pandas.core.reshape.pivot.crosstab partial support

  • PNDSPY1167: pandas.core.reshape.pivot.pivot partial support

  • PNDSPY1168: pandas.core.reshape.pivot.pivot_table partial support

  • PNDSPY1169: pandas.core.reshape.tile.cut partial support

  • PNDSPY1170: pandas.core.reshape.tile.qcut partial support

  • PNDSPY1171: pandas.core.series.Series.add partial support

  • PNDSPY1172: pandas.core.series.Series.all partial support

  • PNDSPY1173: pandas.core.series.Series.any partial support

  • PNDSPY1174: pandas.core.series.Series.case_when partial support

  • PNDSPY1175: pandas.core.series.Series.compare partial support

  • PNDSPY1176: pandas.core.series.Series.cumsum partial support

  • PNDSPY1177: pandas.core.series.Series.div partial support

  • PNDSPY1178: pandas.core.series.Series.divide partial support

  • PNDSPY1179: pandas.core.series.Series.dropna partial support

  • PNDSPY1180: pandas.core.series.Series.eq partial support

  • PNDSPY1181: pandas.core.series.Series.flags partial support

  • PNDSPY1182: pandas.core.series.Series.floordiv partial support

  • PNDSPY1183: pandas.core.series.Series.ge partial support

  • PNDSPY1184: pandas.core.series.Series.groupby partial support

  • PNDSPY1185: pandas.core.series.Series.gt partial support

  • PNDSPY1186: pandas.core.series.Series.le partial support

  • PNDSPY1187: pandas.core.series.Series.lt partial support

  • PNDSPY1188: pandas.core.series.Series.map partial support

  • PNDSPY1189: pandas.core.series.Series.mod partial support

  • PNDSPY1190: pandas.core.series.Series.mul partial support

  • PNDSPY1191: pandas.core.series.Series.multiply partial support

  • PNDSPY1192: pandas.core.series.Series.ne partial support

  • PNDSPY1193: pandas.core.series.Series.nlargest partial support

  • PNDSPY1194: pandas.core.series.Series.nsmallest partial support

  • PNDSPY1195: pandas.core.series.Series.pow partial support

  • PNDSPY1196: pandas.core.series.Series.quantile partial support

  • PNDSPY1197: pandas.core.series.Series.radd partial support

  • PNDSPY1198: pandas.core.series.Series.rdiv partial support

  • PNDSPY1199: pandas.core.series.Series.reindex partial support

  • PNDSPY1200: pandas.core.series.Series.rename partial support

  • PNDSPY1201: pandas.core.series.Series.rfloordiv partial support

  • PNDSPY1202: pandas.core.series.Series.rmod partial support

  • PNDSPY1203: pandas.core.series.Series.rmul partial support

  • PNDSPY1204: pandas.core.series.Series.rpow partial support

  • PNDSPY1205: pandas.core.series.Series.rsub partial support

  • PNDSPY1206: pandas.core.series.Series.rtruediv partial support

  • PNDSPY1207: pandas.core.series.Series.skew partial support

  • PNDSPY1208: pandas.core.series.Series.sort_index partial support

  • PNDSPY1209: pandas.core.series.Series.sort_values partial support

  • PNDSPY1210: pandas.core.series.Series.std partial support

  • PNDSPY1211: pandas.core.series.Series.sub partial support

  • PNDSPY1212: pandas.core.series.Series.subtract partial support

  • PNDSPY1213: pandas.core.series.Series.truediv partial support

  • PNDSPY1214: pandas.core.series.Series.unstack partial support

  • PNDSPY1215: pandas.core.series.Series.var partial support

  • PNDSPY1216: pandas.core.strings.accessor.StringMethods._*getitem*_ partial support

  • PNDSPY1217: pandas.core.strings.accessor.StringMethods.contains partial support

  • PNDSPY1218: pandas.core.strings.accessor.StringMethods.endswith partial support

  • PNDSPY1219: pandas.core.strings.accessor.StringMethods.get partial support

  • PNDSPY1220: pandas.core.strings.accessor.StringMethods.isdigit partial support

  • PNDSPY1221: pandas.core.strings.accessor.StringMethods.len partial support

  • PNDSPY1222: pandas.core.strings.accessor.StringMethods.lstrip partial support

  • PNDSPY1223: pandas.core.strings.accessor.StringMethods.replace partial support

  • PNDSPY1224: pandas.core.strings.accessor.StringMethods.rstrip partial support

  • PNDSPY1225: pandas.core.strings.accessor.StringMethods.slice partial support

  • PNDSPY1226: pandas.core.strings.accessor.StringMethods.split partial support

  • PNDSPY1227: pandas.core.strings.accessor.StringMethods.startswith partial support

  • PNDSPY1228: pandas.core.strings.accessor.StringMethods.strip partial support

  • PNDSPY1229: pandas.core.strings.accessor.StringMethods.translate partial support

  • PNDSPY1230: pandas.core.tools.datetimes.to_datetime partial support

  • PNDSPY1231: pandas.core.tools.numeric.to_numeric partial support

  • PNDSPY1232: pandas.core.tools.timedeltas.to_timedelta partial support

  • PNDSPY1233: pandas.core.window.ewm.ExponentialMovingWindow.corr partial support

  • PNDSPY1234: pandas.core.window.ewm.ExponentialMovingWindow.mean partial support

  • PNDSPY1235: pandas.core.window.ewm.ExponentialMovingWindow.std partial support

  • PNDSPY1236: pandas.core.window.ewm.ExponentialMovingWindow.sum partial support

  • PNDSPY1237: pandas.core.window.ewm.ExponentialMovingWindow.var partial support

  • PNDSPY1238: pandas.core.window.expanding.Expanding.corr partial support

  • PNDSPY1239: pandas.core.window.expanding.Expanding.count partial support

  • PNDSPY1240: pandas.core.window.expanding.Expanding.max partial support

  • PNDSPY1241: pandas.core.window.expanding.Expanding.mean partial support

  • PNDSPY1242: pandas.core.window.expanding.Expanding.min partial support

  • PNDSPY1243: pandas.core.window.expanding.Expanding.sem partial support

  • PNDSPY1244: pandas.core.window.expanding.Expanding.std partial support

  • PNDSPY1245: pandas.core.window.expanding.Expanding.sum partial support

  • PNDSPY1246: pandas.core.window.expanding.Expanding.var partial support

  • PNDSPY1247: pandas.core.window.rolling.Rolling.corr partial support

  • PNDSPY1248: pandas.core.window.rolling.Rolling.count partial support

  • PNDSPY1249: pandas.core.window.rolling.Rolling.max partial support

  • PNDSPY1250: pandas.core.window.rolling.Rolling.mean partial support

  • PNDSPY1251: pandas.core.window.rolling.Rolling.min partial support

  • PNDSPY1252: pandas.core.window.rolling.Rolling.sem partial support

  • PNDSPY1253: pandas.core.window.rolling.Rolling.std partial support

  • PNDSPY1254: pandas.core.window.rolling.Rolling.sum partial support

  • PNDSPY1255: pandas.core.window.rolling.Rolling.var partial support

  • PNDSPY1256: pandas.core.window.rolling.Window.mean partial support

  • PNDSPY1257: pandas.core.window.rolling.Window.std partial support

  • PNDSPY1258: pandas.core.window.rolling.Window.sum partial support

  • PNDSPY1259: pandas.core.window.rolling.Window.var partial support

  • PNDSPY1260: pandas.io.json._json.read_json partial support

  • PNDSPY1261: pandas.io.parquet.read_parquet partial support

  • PNDSPY1262: pandas.io.parsers.readers.read_csv partial support

Modifications¶

• Updated the sfutils library implementation to support multiple levels of notebooks calls

• Upgraded supported Snowpark Python version from v1.41.0 to v1.43.0. This upgrade includes the following mapping status changes:

  NotSupported → Direct (8 functions):

  • pyspark.sql.functions.bool_and → snowflake.snowpark.functions.booland_agg

  • pyspark.sql.functions.bucket → snowflake.snowpark.functions.bucket

  • pyspark.sql.functions.cot → snowflake.snowpark.functions.cot

  • pyspark.sql.functions.day → snowflake.snowpark.functions.day

  • pyspark.sql.functions.every → snowflake.snowpark.functions.booland_agg

  • pyspark.sql.functions.pi → snowflake.snowpark.functions.pi

  • pyspark.sql.functions.width_bucket → snowflake.snowpark.functions.width_bucket

  • pyspark.sql.functions.zeroifnull → snowflake.snowpark.functions.zeroifnull

NotSupported → Rename (1 function):

• pyspark.sql.functions.uuid → snowflake.snowpark.functions.uuid_string

• Upgraded supported Snowpark Pandas version from v1.41.0 to v1.43.0.

• The mapping status of the following Pandas elements were updated:

  NotSupported → Direct (56 functions):

  • pandas.core.arrays.datetimes.DatetimeArray.date

  • pandas.core.arrays.datetimes.DatetimeArray.normalize

  • pandas.core.arrays.datetimes.DatetimeArray.time

  • pandas.core.base.IndexOpsMixin.T

  • pandas.core.base.IndexOpsMixin.empty

  • pandas.core.base.IndexOpsMixin.is_monotonic_decreasing

  • pandas.core.base.IndexOpsMixin.is_monotonic_increasing

  • pandas.core.base.IndexOpsMixin.is_unique

  • pandas.core.base.IndexOpsMixin.item

  • pandas.core.base.IndexOpsMixin.ndim

  • pandas.core.base.IndexOpsMixin.nunique

  • pandas.core.base.IndexOpsMixin.shape

  • pandas.core.base.IndexOpsMixin.size

  • pandas.core.base.IndexOpsMixin.to_list

  • pandas.core.base.IndexOpsMixin.to_numpy

  • pandas.core.base.IndexOpsMixin.tolist

  • pandas.core.base.IndexOpsMixin.transpose

  • pandas.core.generic.NDFrame.abs

  • pandas.core.generic.NDFrame.add_prefix

  • pandas.core.generic.NDFrame.add_suffix

  • pandas.core.generic.NDFrame.attrs

  • pandas.core.generic.NDFrame.copy

  • pandas.core.generic.NDFrame.describe

  • pandas.core.generic.NDFrame.dtypes

  • pandas.core.generic.NDFrame.equals

  • pandas.core.generic.NDFrame.first

  • pandas.core.generic.NDFrame.first_valid_index

  • pandas.core.generic.NDFrame.get

  • pandas.core.generic.NDFrame.head

  • pandas.core.generic.NDFrame.keys

  • pandas.core.generic.NDFrame.last

  • pandas.core.generic.NDFrame.last_valid_index

  • pandas.core.generic.NDFrame.ndim

  • pandas.core.generic.NDFrame.size

  • pandas.core.generic.NDFrame.squeeze

  • pandas.core.generic.NDFrame.tail

  • pandas.core.generic.NDFrame.take

  • pandas.core.generic.NDFrame.to_excel

  • pandas.core.groupby.groupby.BaseGroupBy.groups

  • pandas.core.groupby.groupby.GroupBy.count

  • pandas.core.groupby.groupby.GroupBy.cumcount

  • pandas.core.groupby.groupby.GroupBy.cummax

  • pandas.core.groupby.groupby.GroupBy.cummin

  • pandas.core.groupby.groupby.GroupBy.cumsum

  • pandas.core.groupby.groupby.GroupBy.head

  • pandas.core.groupby.groupby.GroupBy.max

  • pandas.core.groupby.groupby.GroupBy.mean

  • pandas.core.groupby.groupby.GroupBy.median

  • pandas.core.groupby.groupby.GroupBy.min

  • pandas.core.groupby.groupby.GroupBy.rank

  • pandas.core.groupby.groupby.GroupBy.size

  • pandas.core.groupby.groupby.GroupBy.tail

  • pandas.core.indexes.datetimes.DatetimeIndex.year

  • pandas.core.indexing.IndexingMixin.iat

  • pandas.core.indexing.IndexingMixin.iloc

  • pandas.core.series.Series.first

NotSupported → Partial (70 functions):

• pandas.core.arrays.datetimelike.DatelikeOps.strftime (PNDSPY1019)

• pandas.core.arrays.datetimelike.TimelikeOps.ceil (PNDSPY1020)

• pandas.core.arrays.datetimelike.TimelikeOps.floor (PNDSPY1021)

• pandas.core.arrays.datetimelike.TimelikeOps.round (PNDSPY1022)

• pandas.core.arrays.datetimes.DatetimeArray.day_name (PNDSPY1023)

• pandas.core.arrays.datetimes.DatetimeArray.month_name (PNDSPY1024)

• pandas.core.arrays.datetimes.DatetimeArray.tz_convert (PNDSPY1025)

• pandas.core.arrays.datetimes.DatetimeArray.tz_localize (PNDSPY1026)

• pandas.core.base.IndexOpsMixin.argmax (PNDSPY1027)

• pandas.core.base.IndexOpsMixin.argmin (PNDSPY1028)

• pandas.core.base.IndexOpsMixin.value_counts (PNDSPY1029)

• pandas.core.frame.DataFrame.eval (PNDSPY1049)

• pandas.core.frame.DataFrame.expanding (PNDSPY1050)

• pandas.core.frame.DataFrame.melt (PNDSPY1067)

• pandas.core.frame.DataFrame.pct_change (PNDSPY1077)

• pandas.core.frame.DataFrame.quantile (PNDSPY1081)

• pandas.core.frame.DataFrame.std (PNDSPY1103)

• pandas.core.generic.NDFrame.asfreq (PNDSPY1037)

• pandas.core.generic.NDFrame.fillna (PNDSPY1052)

• pandas.core.generic.NDFrame.mask (PNDSPY1066)

• pandas.core.generic.NDFrame.pct_change (PNDSPY1077)

• pandas.core.generic.NDFrame.rank (PNDSPY1083)

• pandas.core.generic.NDFrame.replace (PNDSPY1087)

• pandas.core.generic.NDFrame.shift (PNDSPY1115)

• pandas.core.generic.NDFrame.to_csv (PNDSPY1106)

• pandas.core.generic.NDFrame.tz_convert (PNDSPY1110)

• pandas.core.generic.NDFrame.tz_localize (PNDSPY1111)

• pandas.core.generic.NDFrame.where (PNDSPY1114)

• pandas.core.groupby.generic.DataFrameGroupBy.transform (PNDSPY1121)

• pandas.core.groupby.generic.DataFrameGroupBy.value_counts (PNDSPY1122)

• pandas.core.groupby.groupby.BaseGroupBy.get_group (PNDSPY1123)

• pandas.core.groupby.groupby.GroupBy.bfill (PNDSPY1127)

• pandas.core.groupby.groupby.GroupBy.first (PNDSPY1129)

• pandas.core.groupby.groupby.GroupBy.last (PNDSPY1130)

• pandas.core.groupby.groupby.GroupBy.quantile (PNDSPY1132)

• pandas.core.groupby.groupby.GroupBy.resample (PNDSPY1133)

• pandas.core.groupby.groupby.GroupBy.rolling (PNDSPY1134)

• pandas.core.groupby.groupby.GroupBy.shift (PNDSPY1135)

• pandas.core.groupby.groupby.GroupBy.std (PNDSPY1136)

• pandas.core.groupby.groupby.GroupBy.var (PNDSPY1137)

• pandas.core.indexes.base.Index.nlevels (PNDSPY1140)

• pandas.core.indexes.base.Index.sort_values (PNDSPY1142)

• pandas.core.indexing.IndexingMixin.at (PNDSPY1039)

• pandas.core.indexing.IndexingMixin.loc (PNDSPY1063)

• pandas.core.resample.Resampler.ffill (PNDSPY1155)

• pandas.core.resample.Resampler.first (PNDSPY1157)

• pandas.core.resample.Resampler.last (PNDSPY1158)

• pandas.core.resample.Resampler.std (PNDSPY1160)

• pandas.core.resample.Resampler.var (PNDSPY1161)

• pandas.core.reshape.merge.merge_asof (PNDSPY1165)

• pandas.core.reshape.pivot.pivot (PNDSPY1167)

• pandas.core.series.Series.expanding (PNDSPY1050)

• pandas.core.series.Series.pct_change (PNDSPY1077)

• pandas.core.window.ewm.ExponentialMovingWindow.corr (PNDSPY1233)

• pandas.core.window.ewm.ExponentialMovingWindow.mean (PNDSPY1234)

• pandas.core.window.ewm.ExponentialMovingWindow.std (PNDSPY1235)

• pandas.core.window.ewm.ExponentialMovingWindow.sum (PNDSPY1236)

• pandas.core.window.ewm.ExponentialMovingWindow.var (PNDSPY1237)

• pandas.core.window.expanding.Expanding.corr (PNDSPY1238)

• pandas.core.window.expanding.Expanding.max (PNDSPY1240)

• pandas.core.window.expanding.Expanding.mean (PNDSPY1241)

• pandas.core.window.expanding.Expanding.min (PNDSPY1242)

• pandas.core.window.expanding.Expanding.sem (PNDSPY1243)

• pandas.core.window.expanding.Expanding.std (PNDSPY1244)

• pandas.core.window.expanding.Expanding.sum (PNDSPY1245)

• pandas.core.window.expanding.Expanding.var (PNDSPY1246)

• pandas.core.window.rolling.Window.mean (PNDSPY1256)

• pandas.core.window.rolling.Window.std (PNDSPY1257)

• pandas.core.window.rolling.Window.sum (PNDSPY1258)

• pandas.core.window.rolling.Window.var (PNDSPY1259)

(new) → Direct (74 functions):

• pandas.core.arrays.datetimes.DatetimeArray.day

• pandas.core.arrays.datetimes.DatetimeArray.day_of_week

• pandas.core.arrays.datetimes.DatetimeArray.day_of_year

• pandas.core.arrays.datetimes.DatetimeArray.dayofweek

• pandas.core.arrays.datetimes.DatetimeArray.dayofyear

• pandas.core.arrays.datetimes.DatetimeArray.days_in_month

• pandas.core.arrays.datetimes.DatetimeArray.daysinmonth

• pandas.core.arrays.datetimes.DatetimeArray.hour

• pandas.core.arrays.datetimes.DatetimeArray.is_leap_year

• pandas.core.arrays.datetimes.DatetimeArray.is_month_end

• pandas.core.arrays.datetimes.DatetimeArray.is_month_start

• pandas.core.arrays.datetimes.DatetimeArray.is_quarter_end

• pandas.core.arrays.datetimes.DatetimeArray.is_quarter_start

• pandas.core.arrays.datetimes.DatetimeArray.is_year_end

• pandas.core.arrays.datetimes.DatetimeArray.is_year_start

• pandas.core.arrays.datetimes.DatetimeArray.isocalendar

• pandas.core.arrays.datetimes.DatetimeArray.microsecond

• pandas.core.arrays.datetimes.DatetimeArray.minute

• pandas.core.arrays.datetimes.DatetimeArray.month

• pandas.core.arrays.datetimes.DatetimeArray.nanosecond

• pandas.core.arrays.datetimes.DatetimeArray.quarter

• pandas.core.arrays.datetimes.DatetimeArray.second

• pandas.core.arrays.datetimes.DatetimeArray.weekday

• pandas.core.arrays.datetimes.DatetimeArray.year

• pandas.core.arrays.timedeltas.TimedeltaArray.days

• pandas.core.arrays.timedeltas.TimedeltaArray.microseconds

• pandas.core.arrays.timedeltas.TimedeltaArray.nanoseconds

• pandas.core.arrays.timedeltas.TimedeltaArray.seconds

• pandas.core.frame.DataFrame.flags

• pandas.core.generic.NDFrame.flags

• pandas.core.generic.NDFrame.rename_axis

• pandas.core.groupby.groupby.BaseGroupBy.__iter__

• pandas.core.groupby.groupby.BaseGroupBy.__len__

• pandas.core.groupby.groupby.GroupBy.sum

• pandas.core.indexes.base.Index.T

• pandas.core.indexes.datetimes.DatetimeIndex.date

• pandas.core.indexes.datetimes.DatetimeIndex.day

• pandas.core.indexes.datetimes.DatetimeIndex.day_of_week

• pandas.core.indexes.datetimes.DatetimeIndex.day_of_year

• pandas.core.indexes.datetimes.DatetimeIndex.dayofweek

• pandas.core.indexes.datetimes.DatetimeIndex.dayofyear

• pandas.core.indexes.datetimes.DatetimeIndex.hour

• pandas.core.indexes.datetimes.DatetimeIndex.is_month_end

• pandas.core.indexes.datetimes.DatetimeIndex.is_month_start

• pandas.core.indexes.datetimes.DatetimeIndex.mean

• pandas.core.indexes.datetimes.DatetimeIndex.microsecond

• pandas.core.indexes.datetimes.DatetimeIndex.minute

• pandas.core.indexes.datetimes.DatetimeIndex.month

• pandas.core.indexes.datetimes.DatetimeIndex.nanosecond

• pandas.core.indexes.datetimes.DatetimeIndex.normalize

• pandas.core.indexes.datetimes.DatetimeIndex.quarter

• pandas.core.indexes.datetimes.DatetimeIndex.second

• pandas.core.indexes.timedeltas.TimedeltaIndex.total_seconds

• pandas.core.series.Series.info (PNDSPY1018)

• pandas.core.series.Series.tolist

• pandas.core.strings.accessor.StringMethods.capitalize

• pandas.core.strings.accessor.StringMethods.center

• pandas.core.strings.accessor.StringMethods.count

• pandas.core.strings.accessor.StringMethods.islower

• pandas.core.strings.accessor.StringMethods.istitle

• pandas.core.strings.accessor.StringMethods.isupper

• pandas.core.strings.accessor.StringMethods.ljust

• pandas.core.strings.accessor.StringMethods.lower

• pandas.core.strings.accessor.StringMethods.match

• pandas.core.strings.accessor.StringMethods.pad

• pandas.core.strings.accessor.StringMethods.rjust

• pandas.core.strings.accessor.StringMethods.title

• pandas.core.strings.accessor.StringMethods.upper

• snowpark_pandas.read_snowflake

• snowpark_pandas.to_dynamic_table

• snowpark_pandas.to_iceberg

• snowpark_pandas.to_pandas

• snowpark_pandas.to_snowflake

• snowpark_pandas.to_view

(new) → Partial (47 functions):

• pandas.core.frame.DataFrame.__dataframe__ (PNDSPY1031)

• pandas.core.frame.DataFrame.pad (PNDSPY1076)

• pandas.core.generic.NDFrame.align (PNDSPY1033)

• pandas.core.generic.NDFrame.astype (PNDSPY1038)

• pandas.core.generic.NDFrame.expanding (PNDSPY1050)

• pandas.core.generic.NDFrame.ffill (PNDSPY1051)

• pandas.core.generic.NDFrame.interpolate (PNDSPY1015)

• pandas.core.generic.NDFrame.pad (PNDSPY1076)

• pandas.core.generic.NDFrame.resample (PNDSPY1088)

• pandas.core.generic.NDFrame.rolling (PNDSPY1092)

• pandas.core.generic.NDFrame.sample (PNDSPY1097)

• pandas.core.groupby.groupby.GroupBy.all (PNDSPY1124)

• pandas.core.groupby.groupby.GroupBy.any (PNDSPY1125)

• pandas.core.groupby.groupby.GroupBy.apply (PNDSPY1126)

• pandas.core.indexes.base.Index.all (PNDSPY1138)

• pandas.core.indexes.base.Index.any (PNDSPY1139)

• pandas.core.indexes.base.Index.reindex (PNDSPY1141)

• pandas.core.indexes.base.Index.value_counts (PNDSPY1029)

• pandas.core.indexes.datetimes.DatetimeIndex.tz_convert (PNDSPY1149)

• pandas.core.indexes.datetimes.DatetimeIndex.tz_localize (PNDSPY1150)

• pandas.core.series.Series.backfill (PNDSPY1040)

• pandas.core.series.Series.bfill (PNDSPY1041)

• pandas.core.series.Series.flags (PNDSPY1181)

• pandas.core.series.Series.pad (PNDSPY1076)

• pandas.core.strings.accessor.StringMethods.__getitem__ (PNDSPY1216)

• pandas.core.strings.accessor.StringMethods.contains (PNDSPY1217)

• pandas.core.strings.accessor.StringMethods.endswith (PNDSPY1218)

• pandas.core.strings.accessor.StringMethods.get (PNDSPY1219)

• pandas.core.strings.accessor.StringMethods.isdigit (PNDSPY1220)

• pandas.core.strings.accessor.StringMethods.len (PNDSPY1221)

• pandas.core.strings.accessor.StringMethods.lstrip (PNDSPY1222)

• pandas.core.strings.accessor.StringMethods.replace (PNDSPY1223)

• pandas.core.strings.accessor.StringMethods.rstrip (PNDSPY1224)

• pandas.core.strings.accessor.StringMethods.slice (PNDSPY1225)

• pandas.core.strings.accessor.StringMethods.split (PNDSPY1226)

• pandas.core.strings.accessor.StringMethods.startswith (PNDSPY1227)

• pandas.core.strings.accessor.StringMethods.strip (PNDSPY1228)

• pandas.core.strings.accessor.StringMethods.translate (PNDSPY1229)

• pandas.core.window.rolling.Rolling.corr (PNDSPY1247)

• pandas.core.window.rolling.Rolling.max (PNDSPY1249)

• pandas.core.window.rolling.Rolling.mean (PNDSPY1250)

• pandas.core.window.rolling.Rolling.min (PNDSPY1251)

• pandas.core.window.rolling.Rolling.sem (PNDSPY1252)

• pandas.core.window.rolling.Rolling.std (PNDSPY1253)

• pandas.core.window.rolling.Rolling.sum (PNDSPY1254)

• pandas.core.window.rolling.Rolling.var (PNDSPY1255)

• pandas.io.json._json.read_json (PNDSPY1260)

Direct → Partial (12 functions):

• pandas.core.frame.DataFrame.T (PNDSPY1030)

• pandas.core.frame.DataFrame.any (PNDSPY1035)

• pandas.core.frame.DataFrame.where (PNDSPY1114)

• pandas.core.groupby.generic.DataFrameGroupBy.agg (PNDSPY1116)

• pandas.core.indexes.datetimes.DatetimeIndex.round (PNDSPY1147)

• pandas.core.reshape.tile.qcut (PNDSPY1170)

• pandas.core.series.Series.astype (PNDSPY1038)

• pandas.core.series.Series.groupby (PNDSPY1184)

• pandas.core.series.Series.le (PNDSPY1186)

• pandas.core.series.Series.loc (PNDSPY1063)

• pandas.io.parquet.read_parquet (PNDSPY1261)

• pandas.io.parsers.readers.read_csv (PNDSPY1262)

Partial → Direct (5 functions):

• pandas.core.indexes.datetimes.DatetimeIndex.is_leap_year

• pandas.core.indexes.datetimes.DatetimeIndex.is_quarter_end

• pandas.core.indexes.datetimes.DatetimeIndex.is_quarter_start

• pandas.core.indexes.datetimes.DatetimeIndex.is_year_end

• pandas.core.indexes.datetimes.DatetimeIndex.is_year_start

Rename → Partial (4 functions):

• pandas.core.frame.DataFrame.divide (PNDSPY1046)

• pandas.core.frame.DataFrame.multiply (PNDSPY1071)

• pandas.core.frame.DataFrame.subtract (PNDSPY1105)

• pandas.core.series.Series.divide (PNDSPY1178)

Correction¶

• Fixed the « How to read through the scores » link on the assessment and conversion results page to ensure it correctly opens the readiness score documentation.

Included SMA Core Version¶

• Snowpark Conversion Core: 8.1.55

Improvements¶

• License-Free Conversion Mode: A license or access code is no longer required to run SMA in Conversion mode.

• Project Options Page: A new Project Options page has been introduced to present the available workflows in the application, including « Code Analysis and Conversion ».

• Technical Discovery Relocation: The Technical Discovery section has been moved to the Project Creation page for a more streamlined project setup experience.

• Simplified Conversion Setup: The Conversion Setup page has been updated and no longer requires a license or access code.

• Project File Extension: The project file extension has changed from .snowma to .snowct.

• Updated User Interface: The user interface has been refreshed to align with the SnowConvert AI look and feel.

Included SMA Core Version¶

• Snowpark Conversion Core: 8.1.55

Ajouté¶

• Added SQL Language to the DetailedReport doc file.

• Added SQL configuration cell at the beginning of a converted Databricks-to-Jupyter transformation to be compatible with Snowflake notebooks.

Modifications¶

• Updated the %run magic command transformation to append .ipynb extension to notebook paths.

  • For unquoted paths: %run ./myNotebook transforms to %run ./myNotebook.ipynb

  • For quoted paths: %run "./myNotebook" transforms to %run "./myNotebook.ipynb"

• Scala code in notebook cells will now be commented in a python cell during a notebook migration.

• Updated the conversion of dbutils.run to the sfutils.notebook.run function to handle notebook execution calls.

• Bumped the supported versions of Snowpark Python API and Snowpark Pandas API from 1.40.0 to 1.41.0.

• Updated the mapping status for the following Pandas functions from NotSupported to Partial:

  • pandas.core.frame.DataFrame.agg → modin.pandas.DataFrame.agg

  • pandas.core.frame.DataFrame.interpolate → modin.pandas.DataFrame.interpolate

  • pandas.core.reshape.encoding.get_dummies → modin.pandas.general.get_dummies

  • pandas.core.series.Series.agg → modin.pandas.Series.agg

  • pandas.core.series.Series.interpolate → modin.pandas.Series.interpolate

Correction¶

• SMA now will rename .hql (Hive SQL) files to .sql after conversion.

• The implicit cell for a DBX Scala Notebook when converting to Snowflake will be a python cell with an EWI. The Scala code will be commented out.

• Python cells from DBX SQL Notebooks will preserve the language metadata.

Supprimée¶

• Removed the previous %run transformation in DBX notebooks that generated spark.sql("EXECUTE NOTEBOOK ...") SQL statements.

• The SnowConvert MissingObjects report was absorbed by the MissingObjectReference report. The MissingObjects report will no longer be generated.

Included SMA Core Version¶

• Snowpark Conversion Core: 8.1.43

Included SnowConvert AI Version¶

• SnowConvert AI Version 2.2.0 (Release Notes)

Ajouté¶

• Enhanced Notebook Setup for Assessment: When running an assessment on Databricks notebooks, a Snowpark Connect session is now automatically added to the first cell to simplify your setup.

• Automatic Snowpark Connect Conversion: The tool now automatically converts both SparkSession and SparkContext initializations in Python code to their equivalent Snowpark Connect sessions.

• Improved Error Identification:

  • Added a new warning code, SPRKCNTPY4000, to clearly flag any SparkContext elements that are not yet supported by Snowpark Connect.

  • The tool now automatically detects and flags unsupported Databricks utility calls (dbutils API) with the new warning code SPRKDBX1004 during conversion.

• More Detailed Reporting:

  • The SparkUsagesInventory.csv report now includes a new column called IS_SNOWPARK_CONNECT_TOOL_SUPPORTED

  • This new column is to clearly indicate if a Spark element is supported directly by Snowpark Connect, or supported throught an SMA transformation.

  • The Snowpark Connect readiness score calculation has been updated to use the new IS_SNOWPARK_CONNECT_TOOL_SUPPORTED column in the SparkUsagesInventory.csv report.

• Next-Generation Notebook Support: Enhanced support for the VNext Snowflake Notebooks format when converting Databricks or Jupyter notebooks.

  • Full VNext Compatibility: The SMA can now generate output files that fully adhere to the VNext Snowflake Notebooks standard, regardless of whether the source was a Databricks or a previous-generation Jupyter notebook.

  • Smarter Language Handling: The conversion engine has been updated with enhanced logic to accurately detect and manage the specific language (such as Python or Scala) within each individual notebook cell. This allows for more precise and reliable cell-by-cell conversion.

  • Enhanced Metadata for Cells: The process now correctly incorporates necessary language and type metadata at the cell level during generation, which is essential for VNext Notebooks to function as expected.

Modifications¶

• Simplified Python Code: For Snowpark Connect, unnecessary .sparkContext references in Python method calls are now removed to streamline your code.

• Clearer Warning Codes: Snowpark Connect warning codes are now renamed to include language-specific prefixes (e.g., SPRKCNTPY for Python, SPRKCNTSCL for Scala) for easier error identification.

• More Accurate Notebook Conversions: The conversion process for notebooks has been improved to correctly distinguish between Databricks and Jupyter formats, preventing incorrect modifications.

Correction¶

• Fixed a bug in the artifact dependency inventory that incorrectly reported .options() configuration as a data source.

Ajouté¶

• Technical Discovery View: A new Technical Discovery View is now available in the desktop application.

• SMA Assessment AI: SMA desktop application is now directly integrated with an optional LLM interface.

  • Ask questions about your assessment results

  • Get help with how to approach the migration

  • Connect and deploy your assessment results directly into your Snowflake account.

Modifications¶

• The Command Line Interface (CLI) parameter for controlling Jupyter conversion has been updated from --enableJupyter to --disableJupyterConversion for clearer functionality.

Versions de base de SMA incluses¶

• Snowpark Conversion Core: 8.1.26

Included SnowConvert AI Version¶

• SnowConvert AI Version 2.0.57 (Release Notes: SnowConvert AI - Recent Release Notes | Snowflake Documentation)

Ajouté¶

• The Execution Summary section of the DetailedReport.docx now indicates whether the SMA was run in Assessment or Conversion mode.

Modifications¶

• Bumped the supported versions of Snowpark Python API and Snowpark Pandas API from 1.39.0 to 1.40.0.

PySpark Function Mapping Updates:

NotSupported to Rename:

• pyspark.sql.functions.unhex → snowflake.snowpark.functions.hex_decode_binary

Direct to Rename:

• pyspark.sql.functions.greatest → snowflake.snowpark.functions.greatest_ignore_nulls

• pyspark.sql.functions.least → snowflake.snowpark.functions.least_ignore_nulls

NotDefined to Rename:

• pyspark.sql.functions.bool_or → snowflake.snowpark.functions.boolor_agg

• pyspark.sql.functions.char → snowflake.snowpark.functions.chr

NotDefined to Direct:

• pyspark.sql.functions.nullif → snowflake.snowpark.functions.nullif

• pyspark.sql.functions.nvl2 → snowflake.snowpark.functions.nvl2

Snowpark Pandas Function Mapping Updates:

NotSupported to Partial:

• modin.pandas.DataFrame.query → snowflake.snowpark.pandas.core.frame.DataFrame.query

• Added a new EWI PNDSPY1012 to indicate that modin.pandas.DataFrame.query does not support MultiIndex. The following example scenario illustrating this limitation is also included in the EWI documentation.

  CopyExpand

  from snowflake.snowpark.modin import plugin
  import modin.pandas as pd # Snowpark pandas
  
  # Create a DataFrame with single-level index
  data = {
      'name': ['Alice', 'Bob', 'Charlie', 'David', 'Eve', 'Frank'],
      'age': [25, 30, 35, 28, 32, 45],
      'salary': [50000, 60000, 75000, 55000, 80000, 90000],
      'department': ['Sales', 'IT', 'HR', 'Sales', 'IT', 'HR']
  }
  df = pd.DataFrame(data)
  
  # Set a single-level index
  df = df.set_index('name')
  print("DataFrame with single-level index:")
  print(df)
  
  # Use query() - This works fine!
  #EWI: PNDSPY1012 => pandas.core.frame.DataFrame.query does not support DataFrames that have a row MultiIndex. Check Snowpark Pandas documentation for more details.
  result = df.query("age > 30 and salary < 85000")
  
  # Create a DataFrame with MultiIndex on rows
  data = {
      'A': [1, 2, 3, 4, 5, 6],
      'B': [10, 20, 30, 40, 50, 60],
      'C': ['x', 'y', 'x', 'y', 'x', 'y']
  }
  df = pd.DataFrame(data)
  
  # Create MultiIndex
  df = df.set_index([
      pd.Index(['group1', 'group1', 'group2', 'group2', 'group3', 'group3']),
      pd.Index(['a', 'b', 'a', 'b', 'a', 'b'])
  ])
  df.index.names = ['group', 'subgroup']
  
  # This will ERROR in Snowpark pandas!
  #EWI: PNDSPY1012 => pandas.core.frame.DataFrame.query does not support DataFrames that have
  

  Show lessSee more

  Scroll to top

  Recommended fix: If the DataFrame contains a MultiIndex, it is necessary to validate the behavior of the query() method in Snowpark pandas. Ensure that the DataFrame structure is compatible with Snowpark pandas” limitations, as MultiIndex rows are not supported. Consider restructuring the DataFrame to use a single-level index or alternative filtering methods.

• Updated all documentation links in the DetailedReport.docx to point to the official Snowflake documentation, replacing the legacy Snowpark Migration Accelerator site.

• Updated the Snowpark Connect readiness score descriptions in the DetailedReport.docx to match the SMA UI.

• Usages of pyspark.sql.window.WindowSpec.orderBy are now reported as supported by Snowpark Connect.

Correction¶

• Fixed broken internal links in the DetailedReport.docx to ensure proper navigation between document sections.

• Added a CellId column to the issues inventory to easily identify the location of EWIs within notebook files.

Versions de base de SMA incluses¶

• Snowpark Conversion Core: 8.1.8

Correction¶

• Fixed an issue where the SMA generated corrupted Databricks notebook files in the output directory during Assessment mode execution.

• Fixed an issue where the SMA would crash if the input directory contained folders named “SMA_ConvertedNotebooks”.

Versions de base de SMA incluses¶

• Snowpark Conversion Core: 8.1.7

Ajouté¶

• Added the Snowpark Connect readiness score. This new score measures the percentage of Spark API references in your codebase that are supported by Snowpark Connect for Spark.

  • This will now be the only score shown in assessment mode. To generate the Snowpark API Readiness Score, run the SMA in conversion mode.

• Added support for SQL embedded migration for literal string concatenations assigned to a local variable in the same scope of execution.

  • Included scenarios now include: .. code-block:: python

    sqlStat = « SELECT colName «  + « FROM myTable » session.sql(sqlStat)

Modifications¶

• Updated the EWI URLs in the Issues.csv inventory to point to the main Snowflake documentation site.

Correction¶

• Fixed a code issue that caused inner project configuration files (e.g., pom.xml, build.sbt, build.gradle) to be incorrectly placed in the root of the output directory instead of the correct inner directories after migration.

Ajouté¶

• Added the Snowpark Connect readiness score and updated the assessment execution flow.

  • When running the application in assessment mode, only the Snowpark Connect readiness score is now displayed.

  • When running the application in conversion mode, the Snowpark API readiness score is displayed (the Snowpark Connect Readiness will not be shown).

Modifications¶

Updated all in-application documentation links to point to the official Snowflake documentation, replacing the legacy SnowConvert site.

Versions de base de SMA incluses¶

• Snowpark Conversion Core 8.0.73

Correction¶

• Fixed an issue where the Snowpark Migration Accelerator failed converting DBC files into Jupyter Notebooks properly.

Versions de base de SMA incluses¶

• Snowpark Conversion Core 8.0.72

Ajouté¶

• Ajout de la prise en charge de Snowpark Scala v1.17.0 :

De non pris en charge à direct :

Ensemble de données :

• org.apache.spark.sql.Dataset.isEmpty → com.snowflake.snowpark.DataFrame.isEmpty

Ligne :

• org.apache.spark.sql.Row.mkString → com.snowflake.snowpark.Row.mkString

StructType :

• org.apache.spark.sql.types.StructType.fieldNames → com.snowflake.snowpark.types.StructType.fieldNames

De Non pris en charge à Renommer :

Fonctions :

• org.apache.spark.functions.flatten → com.snowflake.snowpark.functions.array_flatten

De Direct à Renommer :

Fonctions :

• org.apache.spark.functions.to_date → com.snowflake.snowpark.functions.try_to_date

• org.apache.spark.functions.to_timestamp → com.snowflake.snowpark.functions.try_to_timestamp

D’Assistant direct à Renommer :

Fonctions :

• org.apache.spark.sql.functions.concat_ws → com.snowflake.snowpark.functions.concat_ws_ignore_nulls

De non défini à direct :

Fonctions :

• org.apache.spark.functions.try_to_timestamp → com.snowflake.snowpark.functions.try_to_timestamp

• Le SQL intégré est maintenant migré lorsqu’une instruction SQL littérale est attribuée à une variable locale.

Exemple : sqlStat = “SELECT colName FROM myTable » session.sql(sqlStat)

• Le SQL intégré est désormais pris en charge pour les concaténations de chaînes littérales.

Exemple : session.sql(“SELECT colName «  + « FROM myTable »)

Modifications¶

• Mise à jour des versions prises en charge de Snowpark Python API et Snowpark Pandas API de 1.36.0 à 1.39.0.

• Mise à jour de l’état du mappage pour les fonctions xpath PySpark suivantes de NotSupported à Direct avec l’EWI SPRKPY1103 :

  • pyspark.sql.functions.xpath

  • pyspark.sql.functions.xpath_boolean

  • pyspark.sql.functions.xpath_double

  • pyspark.sql.functions.xpath_float

  • pyspark.sql.functions.xpath_int

  • pyspark.sql.functions.xpath_long

  • pyspark.sql.functions.xpath_number

  • pyspark.sql.functions.xpath_short

  • pyspark.sql.functions.xpath_string

• Mise à jour de l’état du mappage pour les éléments PySpark suivants de NotDefined à Direct :

  • pyspark.sql.functions.bit_and → snowflake.snowpark.functions.bitand_agg

  • pyspark.sql.functions.bit_or → snowflake.snowpark.functions.bitor_agg

  • pyspark.sql.functions.bit_xor → snowflake.snowpark.functions.bitxor_agg

  • pyspark.sql.functions.getbit → snowflake.snowpark.functions.getbit

• Mise à jour de l’état du mappage pour les éléments Pandas suivants de NotSupported à Direct :

  • pandas.core.indexes.base.Index → modin.pandas.Index

  • pandas.core.indexes.base.Index.get_level_values → modin.pandas.Index.get_level_values

• Mise à jour de l’état du mappage pour les fonctions PySpark suivantes de NotSupported à Renommer :

  • pyspark.sql.functions.now → snowflake.snowpark.functions.current_timestamp

Correction¶

• Correction d’un problème empêchant Scala de migrer les importations en cas de renommage.

  Exemple :

  Code source :

  CopyExpand

  package com.example.functions
  import org.apache.spark.sql.functions.{to_timestamp, lit}
  object ToTimeStampTest extends App {
     to_timestamp(lit("sample"))
     to_timestamp(lit("sample"), "yyyy-MM-dd")
   }
  

  Show lessSee more

  Scroll to top

  Code de sortie :

  CopyExpand

  package com.example.functions
  import com.snowflake.snowpark.functions.{try_to_timestamp, lit}
  import com.snowflake.snowpark_extensions.Extensions._
  import com.snowflake.snowpark_extensions.Extensions.functions._
  object ToTimeStampTest extends App {
     try_to_timestamp(lit("sample"))
     try_to_timestamp(lit("sample"), "yyyy-MM-dd")
   }
  

  Show lessSee more

  Scroll to top

Versions de base de SMA incluses¶

• Snowpark Conversion Core 8.0.62

Ajouté¶

• Ajout d’une fonctionnalité permettant de migrer le SQL intégré avec une interpolation au format Python.

• Added support for DataFrame.select and DataFrame.sort transformations for greater data processing flexibility.

Modifications¶

• Mise à jour des versions prises en charge de Snowpark Python API et Snowpark Pandas API vers 1.36.0.

• Updated the mapping status of pandas.core.frame.DataFrame.boxplot from Not Supported to Direct.

• Updated the mapping status of DataFrame.select, Dataset.select, DataFrame.sort and Dataset.sort from Direct to Transformation.

• Snowpark Scala allows a sequence of columns to be passed directly to the select and sort functions, so this transformation changes all the usages such as df.select(cols: _*) to df.select(cols) and df.sort(cols: _*) to df.sort(cols).

• Mise à jour de Python AST et de la version de Parser vers 149.1.9.

• Mise à jour de l’état vers Direct pour les fonctions Pandas suivantes :

  • pandas.core.frame.DataFrame.to_excel

  • pandas.core.series.Series.to_excel

  • pandas.io.feather_format.read_feather

  • pandas.io.orc.read_orc

  • pandas.io.stata.read_stata

• Updated the status for pyspark.sql.pandas.map_ops.PandasMapOpsMixin.mapInPandas to workaround using the EWI SPRKPY1102.

Correction¶

• Correction d’un problème qui affectait les transformations SqlEmbedded lors de l’utilisation d’appels de méthodes chaînés.

• Correction des transformations impliquant PySqlExpr et utilisant le nouveau PyLiteralSql pour éviter de perdre des files d’attente.

• Résolution des problèmes de stabilité interne pour améliorer la robustesse et la fiabilité des outils.

Versions de base de SMA incluses¶

• Snowpark Conversion Core 8.0.46

Ajouté¶

• Ajout de la nouvelle documentation PNDSPY1011 pour l’EWI Pandas.

• Ajout de la prise en charge aux fonctions Pandas suivantes :

  • pandas.core.algorithms.unique

  • pandas.core.dtypes.missing.isna

  • pandas.core.dtypes.missing.isnull

  • pandas.core.dtypes.missing.notna

  • pandas.core.dtypes.missing.notnull

  • pandas.core.resample.Resampler.count

  • pandas.core.resample.Resampler.max

  • pandas.core.resample.Resampler.mean

  • pandas.core.resample.Resampler.median

  • pandas.core.resample.Resampler.min

  • pandas.core.resample.Resampler.size

  • pandas.core.resample.Resampler.sum

  • pandas.core.arrays.timedeltas.TimedeltaArray.total_seconds

  • pandas.core.series.Series.get

  • pandas.core.series.Series.to_frame

  • pandas.core.frame.DataFrame.assign

  • pandas.core.frame.DataFrame.get

  • pandas.core.frame.DataFrame.to_numpy

  • pandas.core.indexes.base.Index.is_unique

  • pandas.core.indexes.base.Index.has_duplicates

  • pandas.core.indexes.base.Index.shape

  • pandas.core.indexes.base.Index.array

  • pandas.core.indexes.base.Index.str

  • pandas.core.indexes.base.Index.equals

  • pandas.core.indexes.base.Index.identical

  • pandas.core.indexes.base.Index.unique

Ajout de la prise en charge aux fonctions Scala suivantes :

• org.apache.spark.sql.functions.format_number

• org.apache.spark.sql.functions.from_unixtime

• org.apache.spark.sql.functions.instr

• org.apache.spark.sql.functions.months_between

• org.apache.spark.sql.functions.pow

• org.apache.spark.sql.functions.to_unix_timestamp

• org.apache.spark.sql.Row.getAs

Modifications¶

• Mise à jour de la version de Snowpark Pandas API prise en charge par SMA vers la version 1.33.0.

• Mise à jour de la version de Snowpark Scala API prise en charge par SMA vers la version 1.16.0.

• Mise à jour de l’état du mappage de pyspark.sql.group.GroupedData.pivot de Transformation à Direct.

• Mise à jour de l’état du mappage de org.apache.spark.sql.Builder.master de NotSupported à Transformation. Cette transformation supprime toutes les utilisations identifiées de cet élément pendant la conversion du code.

• Mise à jour de l’état du mappage de org.apache.spark.sql.types.StructType.fieldIndex de NotSupported à Direct.

• Mise à jour de l’état du mappage de org.apache.spark.sql.Row.fieldIndex de NotSupported à Direct.

• Mise à jour de l’état du mappage de org.apache.spark.sql.SparkSession.stop de NotSupported à Renommer. Toutes les utilisations identifiées de cet élément sont renommées en com.snowflake.snowpark.Session.close lors de la conversion du code.

• Mise à jour de l’état du mappage de org.apache.spark.sql.DataFrame.unpersist et org.apache.spark.sql.Dataset.unpersist de NotSupported à Transformation. Cette transformation supprime toutes les utilisations identifiées de cet élément pendant la conversion du code.

Correction¶

• Correction de la barre oblique inverse de continuation sur les fonctions de file d’attente supprimées.

• Fix the LIBRARY_PREFIX column in the ConversionStatusLibraries.csv file to use the right identifier for scikit-learn library family (scikit-*).

• Correction d’un bogue qui n’analysait pas les opérations groupées à plusieurs lignes.

Ajouté¶

• The following mappings are now performed for org.apache.spark.sql.Dataset[T]:

  • org.apache.spark.sql.Dataset.union is now com.snowflake.snowpark.DataFrame.unionAll

  • org.apache.spark.sql.Dataset.unionByName is now com.snowflake.snowpark.DataFrame.unionAllByName

• Added support for org.apache.spark.sql.functions.broadcast as a transformation.

Modifications¶

• Increased the supported Snowpark Python API version for SMA from 1.27.0 to 1.33.0.

• The status for the pyspark.sql.function.randn function has been updated to Direct.

Correction¶

• Resolved an issue where org.apache.spark.SparkContext.parallelize was not resolving and now supports it as a transformation.

• Fixed the Dataset.persist transformation to work with any type of Dataset, not just Dataset[Row].

Ajouté¶

• Mappages ajustés pour les méthodes spark.DataReader.

• DataFrame.union is now DataFrame.unionAll.

• DataFrame.unionByName is now DataFrame.unionAllByName.

• Ajout de colonnes de dépendances d’artefacts à plusieurs niveaux dans l’inventaire d’artefacts.

• Added new Pandas EWIs documentation, from PNDSPY1005 to PNDSPY1010.

• Added a specific EWI for pandas.core.series.Series.apply.

Modifications¶

• Bumped the version of Snowpark Pandas API supported by the SMA from 1.27.0 to 1.30.0.

Correction¶

• Correction d’un problème avec des valeurs manquantes dans la formule pour obtenir le score de préparation SQL.

• Correction d’un bogue qui provoquait l’affichage du message EWI par défaut de PySpark pour certains éléments Pandas.

Versions de base de SMA incluses¶

• Snowpark Conversion Core 8.0.19

Modifications¶

• Refactored Pandas Imports: Pandas imports now use modin.pandas instead of snowflake.snowpark.modin.pandas.

• Improved dbutils and Magic Commands Transformation:

  • A new sfutils.py file is now generated, and all dbutils prefixes are replaced with sfutils.

  • For Databricks (DBX) notebooks, an implicit import for sfutils is automatically added.

  • The sfutils module simulates various dbutils methods, including file system operations (dbutils.fs) via a defined Snowflake FileSystem (SFFS) stage, and handles notebook execution (dbutils.notebook.run) by transforming it to EXECUTE NOTEBOOK SQL functions.

  • dbutils.notebook.exit is removed as it is not required in Snowflake.

Correction¶

• Updates in SnowConvert Reports: SnowConvert reports now include the CellId column when instances originate from SMA, and the FileName column displays the full path.

• Updated Artifacts Dependency for SnowConvert Reports: The SMA’s artifact inventory report, which was previously impacted by the integration of SnowConvert, has been restored. This update enables the SMA tool to accurately capture and analyze Object References and Missing Object References directly from SnowConvert reports, thereby ensuring the correct retrieval of SQL dependencies for the inventory.

Ajouté¶

• Améliorations apportées aux données télémétriques.

Correction¶

• Correction des liens de documentation dans la fenêtre contextuelle des paramètres de conversion et dans les EWIs Pandas.

Versions de base de SMA incluses¶

• Snowpark Conversion Core 8.0.16

Ajouté¶

• Transformation de Spark XML vers Snowpark

• Option Databricks SQL dans le langage source SQL

• Transformation des connexions de lecture JDBC.

Modifications¶

• Tous les rapports SnowConvert sont copiés dans le fichier Zip de sauvegarde.

• The folder is renamed from SqlReports to SnowConvertReports.

• SqlFunctionsInventory is moved to the folder Reports.

• Tous les rapports SnowConvert sont envoyés vers les données télémétriques.

Correction¶

• Problème non déterministe avec le score de préparation SQL.

• Correction d’un résultat critique faux-positif qui provoquait le plantage du bureau.

• Correction d’un problème qui empêchait le rapport de dépendances d’artefacts d’afficher les objets SQL.

Correction¶

• Correction d’un problème lié à l’exécution de SMA sur le dernier OS Windows, comme indiqué précédemment. Ce correctif résout les problèmes rencontrés dans la version 2.7.1.

Versions de base de SMA incluses¶

• Snowpark Conversion Core 8.0.1

Ajouté¶

The Snowpark Migration Accelerator (SMA) now orchestrates` SnowConvert <https://docs.snowconvert.com/sc/general/about>`_ to process SQL found in user workloads, including embedded SQL in Python / Scala code, Notebook SQL cells, .sql files, and .hql files.

SnowConvert améliore désormais les anciennes capacités de SMA :

• Spark SQL

Un nouveau dossier dans les rapports appelé Rapports SQL contient les rapports générés par SnowConvert.

Problèmes connus¶

La version précédente de SMA pour les rapports SQL apparaîtra vide pour les éléments suivants :

• For Reports/SqlElementsInventory.csv, partially covered by the Reports/SqlReports/Elements.yyyymmdd.hhmmss.csv.

• For Reports/SqlFunctionsInventory.csv refer to the new location with the same name at Reports/SqlReports/SqlFunctionsInventory.csv

L’inventaire des dépendances d’artefacts :

• In the ArtifactDependencyInventory the column for the SQL Object will appear empty

Versions de base de SMA incluses¶

• Snowpark Conversion Core 7.4.0

Correction¶

• Fixed wrong values in the “checkpoints.json” file.

  • La valeur « échantillon » apparaissait sans décimales (pour les valeurs entières) et sans guillemets.

  • La valeur « entryPoint » apparaissait avec des points au lieu de barres obliques et sans extension de fichier.

• Mise à jour de la valeur par défaut sur TRUE pour le paramètre « Convertir les notebooks DBX en notebooks Snowflake »

Application de bureau¶

• Added checkpoints execution settings mechanism recognition.

• Added a mechanism to collect DBX magic commands into DbxElementsInventory.csv

• Added “checkpoints.json” generation into the input directory.

• Added a new EWI for all not supported magic command.

• Added the collection of dbutils into DbxElementsInventory.csv from scala source notebooks

Versions de base de SMA incluses¶

• Snowpark Conversion Core 7.2.53

Modifications¶

• Updates made to handle transformations from DBX Scala elements to Jupyter Python elements, and to comment the entire code from the cell.

• Updates made to handle transformations from dbutils.notebook.run and “r » commands, for the last one, also comment out the entire code from the cell.

• Updated the name and the letter of the key to make the conversion of the notebook files.

Correction¶

• Fixed the bug that was causing the transformation of DBX notebooks into .ipynb files to have the wrong format.

• Fixed the bug that was causing .py DBX notebooks to not be transformable into .ipynb files.

• Fixed a bug that was causing comments to be missing in the output code of DBX notebooks.

• Fixed a bug that was causing raw Scala files to be converted into ipynb files.

Versions de base de SMA incluses¶

• Snowpark Conversion Core 7.2.42

Modifications¶

Mise à jour de DataFramesInventory pour remplir la colonne EntryPoints

Application de bureau¶

Ajouté¶

• Mise à jour du lien DBx EWI dans la page de résultats de l’UI

Versions de base de SMA incluses¶

• Snowpark Conversion Core 7.2.39

Ajouté¶

• Added Execution Flow inventory generation.

• Ajout de la configuration de sessions implicites dans chaque transformation de notebook DBx

Modifications¶

• Le fichier DbUtilsUsagesInventory.csv a été renommé DbxElementsInventory.csv

Correction¶

• Correction d’un bogue qui provoquait une erreur d’analyse lorsqu’une barre oblique inverse apparaissait après une indication de type.

• Correction des importations relatives ne commençant pas par un point et des importations relatives avec un astérisque.

Application de bureau¶

Ajouté¶

• Added a new conversion setting toggle to enable or disable Sma-Checkpoints feature.

• Fix report issue to not crash when post api returns 500

Versions de base de SMA incluses¶

• Snowpark Conversion Core 7.2.26

Ajouté¶

• Added generation of the checkpoints.json file into the output folder based on the DataFramesInventory.csv.

• Added « disableCheckpoints » flag into the CLI commands and additional parameters of the code processor.

• Added a new replacer for Python to transform the dbutils.notebook.run node.

• Added new replacers to transform the magic %run command.

• Added new replacers (Python and Scala) to remove the dbutils.notebook.exit node.

• Added Location column to artifacts inventory.

Modifications¶

• Refonte du séparateur de répertoires normalisé utilisé dans certaines parties de la solution.

• Centralisation de la gestion des noms des dossiers de travail pour l’extraction DBC.

• Updated Snowpark and Pandas version to v1.27.0

• Mise à jour des colonnes de l’inventaire d’artefacts comme suit :

  • Nom -> Dépendance

  • Fichier -> FileId

  • Status -> Status_detail

• Added new column to the artifacts inventory:

  • Success

Correction¶

• Dataframes inventory was not being uploaded to the stage correctly.

Versions de base de SMA incluses¶

• Snowpark Conversion Core 7.2.0

Ajouté¶

• An Artifact Dependency Inventory

• A replacer and EWI for pyspark.sql.types.StructType.fieldNames method to snowflake.snowpark.types.StructType.fieldNames attribute.

• Les fonctions PySpark suivantes avec l’état :

Direct Status

• pyspark.sql.functions.bitmap_bit_position

• pyspark.sql.functions.bitmap_bucket_number

• pyspark.sql.functions.bitmap_construct_agg

• pyspark.sql.functions.equal_null

• pyspark.sql.functions.ifnull

• pyspark.sql.functions.localtimestamp

• pyspark.sql.functions.max_by

• pyspark.sql.functions.min_by

• pyspark.sql.functions.nvl

• pyspark.sql.functions.regr_avgx

• pyspark.sql.functions.regr_avgy

• pyspark.sql.functions.regr_count

• pyspark.sql.functions.regr_intercept

• pyspark.sql.functions.regr_slope

• pyspark.sql.functions.regr_sxx

• pyspark.sql.functions.regr_sxy

• pyspark.sql.functions.regr

NotSupported

• pyspark.sql.functions.map_contains_key

• pyspark.sql.functions.position

• pyspark.sql.functions.regr_r2

• pyspark.sql.functions.try_to_binary

Les fonctions Pandas suivantes avec l’état

• pandas.core.series.Series.str.ljust

• pandas.core.series.Series.str.center

• pandas.core.series.Series.str.pad

• pandas.core.series.Series.str.rjust

Mise à jour des fonctions Pyspark suivantes avec l’état

De WorkAround à Direct

• pyspark.sql.functions.acosh

• pyspark.sql.functions.asinh

• pyspark.sql.functions.atanh

• pyspark.sql.functions.instr

• pyspark.sql.functions.log10

• pyspark.sql.functions.log1p

• pyspark.sql.functions.log2

De NotSupported à Direct

• pyspark.sql.functions.bit_length

• pyspark.sql.functions.cbrt

• pyspark.sql.functions.nth_value

• pyspark.sql.functions.octet_length

• pyspark.sql.functions.base64

• pyspark.sql.functions.unbase64

Mise à jour des fonctions Pandas suivantes avec l’état

De NotSupported à Direct

• pandas.core.frame.DataFrame.pop

• pandas.core.series.Series.between

• pandas.core.series.Series.pop

Versions de base de SMA incluses¶

• Snowpark Conversion Core 7.1.13

Ajouté¶

• Ajout d’une classe de générateur csv pour la création de nouveaux inventaires.

• Added « full_name » column to import usages inventory.

• Added transformation from pyspark.sql.functions.concat_ws to snowflake.snowpark.functions._concat_ws_ignore_nulls.

• Added logic for generation of checkpoints.json.

• Ajout des inventaires :

  • DataFramesInventory.csv.

  • CheckpointsInventory.csv

Application de bureau¶

• Mise à jour du contrat de licence. L’acceptation est nécessaire.

Versions de base de SMA incluses¶

• Snowpark Conversion Core 7.1.2

Ajouté

Updated the mapping status for the following PySpark elements, from NotSupported to Direct

• pyspark.sql.types.ArrayType.json

• pyspark.sql.types.ArrayType.jsonValue

• pyspark.sql.types.ArrayType.simpleString

• pyspark.sql.types.ArrayType.typeName

• pyspark.sql.types.AtomicType.json

• pyspark.sql.types.AtomicType.jsonValue

• pyspark.sql.types.AtomicType.simpleString

• pyspark.sql.types.AtomicType.typeName

• pyspark.sql.types.BinaryType.json

• pyspark.sql.types.BinaryType.jsonValue

• pyspark.sql.types.BinaryType.simpleString

• pyspark.sql.types.BinaryType.typeName

• pyspark.sql.types.BooleanType.json

• pyspark.sql.types.BooleanType.jsonValue

• pyspark.sql.types.BooleanType.simpleString

• pyspark.sql.types.BooleanType.typeName

• pyspark.sql.types.ByteType.json

• pyspark.sql.types.ByteType.jsonValue

• pyspark.sql.types.ByteType.simpleString

• pyspark.sql.types.ByteType.typeName

• pyspark.sql.types.DecimalType.json

• pyspark.sql.types.DecimalType.jsonValue

• pyspark.sql.types.DecimalType.simpleString

• pyspark.sql.types.DecimalType.typeName

• pyspark.sql.types.DoubleType.json

• pyspark.sql.types.DoubleType.jsonValue

• pyspark.sql.types.DoubleType.simpleString

• pyspark.sql.types.DoubleType.typeName

• pyspark.sql.types.FloatType.json

• pyspark.sql.types.FloatType.jsonValue

• pyspark.sql.types.FloatType.simpleString

• pyspark.sql.types.FloatType.typeName

• pyspark.sql.types.FractionalType.json

• pyspark.sql.types.FractionalType.jsonValue

• pyspark.sql.types.FractionalType.simpleString

• pyspark.sql.types.FractionalType.typeName

• pyspark.sql.types.IntegerType.json

• pyspark.sql.types.IntegerType.jsonValue

• pyspark.sql.types.IntegerType.simpleString

• pyspark.sql.types.IntegerType.typeName

• pyspark.sql.types.IntegralType.json

• pyspark.sql.types.IntegralType.jsonValue

• pyspark.sql.types.IntegralType.simpleString

• pyspark.sql.types.IntegralType.typeName

• pyspark.sql.types.LongType.json

• pyspark.sql.types.LongType.jsonValue

• pyspark.sql.types.LongType.simpleString

• pyspark.sql.types.LongType.typeName

• pyspark.sql.types.MapType.json

• pyspark.sql.types.MapType.jsonValue

• pyspark.sql.types.MapType.simpleString

• pyspark.sql.types.MapType.typeName

• pyspark.sql.types.NullType.json

• pyspark.sql.types.NullType.jsonValue

• pyspark.sql.types.NullType.simpleString

• pyspark.sql.types.NullType.typeName

• pyspark.sql.types.NumericType.json

• pyspark.sql.types.NumericType.jsonValue

• pyspark.sql.types.NumericType.simpleString

• pyspark.sql.types.NumericType.typeName

• pyspark.sql.types.ShortType.json

• pyspark.sql.types.ShortType.jsonValue

• pyspark.sql.types.ShortType.simpleString

• pyspark.sql.types.ShortType.typeName

• pyspark.sql.types.StringType.json

• pyspark.sql.types.StringType.jsonValue

• pyspark.sql.types.StringType.simpleString

• pyspark.sql.types.StringType.typeName

• pyspark.sql.types.StructType.json

• pyspark.sql.types.StructType.jsonValue

• pyspark.sql.types.StructType.simpleString

• pyspark.sql.types.StructType.typeName

• pyspark.sql.types.TimestampType.json

• pyspark.sql.types.TimestampType.jsonValue

• pyspark.sql.types.TimestampType.simpleString

• pyspark.sql.types.TimestampType.typeName

• pyspark.sql.types.StructField.simpleString

• pyspark.sql.types.StructField.typeName

• pyspark.sql.types.StructField.json

• pyspark.sql.types.StructField.jsonValue

• pyspark.sql.types.DataType.json

• pyspark.sql.types.DataType.jsonValue

• pyspark.sql.types.DataType.simpleString

• pyspark.sql.types.DataType.typeName

• pyspark.sql.session.SparkSession.getActiveSession

• pyspark.sql.session.SparkSession.version

• pandas.io.html.read_html

• pandas.io.json._normalize.json_normalize

• pyspark.sql.types.ArrayType.fromJson

• pyspark.sql.types.MapType.fromJson

• pyspark.sql.types.StructField.fromJson

• pyspark.sql.types.StructType.fromJson

• pandas.core.groupby.generic.DataFrameGroupBy.pct_change

• pandas.core.groupby.generic.SeriesGroupBy.pct_change

Updated the mapping status for the following Pandas elements, from NotSupported to Direct

• pandas.io.html.read_html

• pandas.io.json._normalize.json_normalize

• pandas.core.groupby.generic.DataFrameGroupBy.pct_change

• pandas.core.groupby.generic.SeriesGroupBy.pct_change

Updated the mapping status for the following PySpark elements, from Rename to Direct

• pyspark.sql.functions.collect_list

• pyspark.sql.functions.size

Correction¶

• Normalisation du format du numéro de version dans les inventaires.

Ajouté¶

Added the following Python Third-Party libraries with Direct status:

• about-time

• affinegap

• aiohappyeyeballs

• alibi-detect

• alive-progress

• allure-nose2

• allure-robotframework

• anaconda-cloud-cli

• anaconda-mirror

• astropy-iers-data

• asynch

• asyncssh

• autots

• autoviml

• aws-msk-iam-sasl-signer-python

• azure-functions

• backports.tarfile

• blas

• bottle

• bson

• cairo

• capnproto

• captum

• categorical-distance

• census

• clickhouse-driver

• clustergram

• cma

• conda-anaconda-telemetry

• configspace

• cpp-expected

• dask-expr

• data-science-utils

• databricks-sdk

• datetime-distance

• db-dtypes

• dedupe

• dedupe-variable-datetime

• dedupe_lehvenshtein_search

• dedupe_levenshtein_search

• diff-cover

• diptest

• dmglib

• docstring_parser

• doublemetaphone

• dspy-ai

• econml

• emcee

• emoji

• environs

• eth-abi

• eth-hash

• eth-typing

• eth-utils

• expat

• filetype

• fitter

• flask-cors

• fpdf2

• frozendict

• gcab

• geojson

• gettext

• glib-tools

• google-ads

• google-ai-generativelanguage

• google-api-python-client

• google-auth-httplib2

• google-cloud-bigquery

• google-cloud-bigquery-core

• google-cloud-bigquery-storage

• google-cloud-bigquery-storage-core

• google-cloud-resource-manager

• google-generativeai

• googlemaps

• grapheme

• graphene

• graphql-relay

• gravis

• greykite

• grpc-google-iam-v1

• harfbuzz

• hatch-fancy-pypi-readme

• haversine

• hiclass

• hicolor-icon-theme

• highered

• hmmlearn

• holidays-ext

• httplib2

• icu

• imbalanced-ensemble

• immutabledict

• importlib-metadata

• importlib-resources

• inquirerpy

• iterative-telemetry

• jaraco.context

• jaraco.test

• jiter

• jiwer

• joserfc

• jsoncpp

• jsonpath

• jsonpath-ng

• jsonpath-python

• kagglehub

• keplergl

• kt-legacy

• langchain-community

• langchain-experimental

• langchain-snowflake

• langchain-text-splitters

• libabseil

• libflac

• libgfortran-ng

• libgfortran5

• libglib

• libgomp

• libgrpc

• libgsf

• libmagic

• libogg

• libopenblas

• libpostal

• libprotobuf

• libsentencepiece

• libsndfile

• libstdcxx-ng

• libtheora

• libtiff

• libvorbis

• libwebp

• lightweight-mmm

• litestar

• litestar-with-annotated-types

• litestar-with-attrs

• litestar-with-cryptography

• litestar-with-jinja

• litestar-with-jwt

• litestar-with-prometheus

• litestar-with-structlog

• lunarcalendar-ext

• matplotlib-venn

• metricks

• mimesis

• modin-ray

• momepy

• mpg123

• msgspec

• msgspec-toml

• msgspec-yaml

• msitools

• multipart

• namex

• nbconvert-all

• nbconvert-core

• nbconvert-pandoc

• nlohmann_json

• numba-cuda

• numpyro

• office365-rest-python-client

• openapi-pydantic

• opentelemetry-distro

• opentelemetry-instrumentation

• opentelemetry-instrumentation-system-metrics

• optree

• osmnx

• pathlib

• pdf2image

• pfzy

• pgpy

• plumbum

• pm4py

• polars

• polyfactory

• poppler-cpp

• postal

• pre-commit

• prompt-toolkit

• propcache

• py-partiql-parser

• py_stringmatching

• pyatlan

• pyfakefs

• pyfhel

• pyhacrf-datamade

• pyiceberg

• pykrb5

• pylbfgs

• pymilvus

• pymoo

• pynisher

• pyomo

• pypdf

• pypdf-with-crypto

• pypdf-with-full

• pypdf-with-image

• pypng

• pyprind

• pyrfr

• pysoundfile

• pytest-codspeed

• pytest-trio

• python-barcode

• python-box

• python-docx

• python-gssapi

• python-iso639

• python-magic

• python-pandoc

• python-zstd

• pyuca

• pyvinecopulib

• pyxirr

• qrcode

• rai-sdk

• ray-client

• ray-observability

• readline

• rich-click

• rouge-score

• ruff

• scikit-criteria

• scikit-mobility

• sentencepiece-python

• sentencepiece-spm

• setuptools-markdown

• setuptools-scm

• setuptools-scm-git-archive

• shareplum

• simdjson

• simplecosine

• sis-extras

• slack-sdk

• smac

• snowflake-sqlalchemy

• snowflake_legacy

• socrata-py

• spdlog

• sphinxcontrib-images

• sphinxcontrib-jquery

• sphinxcontrib-youtube

• splunk-opentelemetry

• sqlfluff

• squarify

• st-theme

• statistics

• streamlit-antd-components

• streamlit-condition-tree

• streamlit-echarts

• streamlit-feedback

• streamlit-keplergl

• streamlit-mermaid

• streamlit-navigation-bar

• streamlit-option-menu

• strictyaml

• stringdist

• sybil

• tensorflow-cpu

• tensorflow-text

• tiledb-ptorchaudio

• torcheval

• trio-websocket

• trulens-connectors-snowflake

• trulens-core

• trulens-dashboard

• trulens-feedback

• trulens-otel-semconv

• trulens-providers-cortex

• tsdownsample

• typing

• typing-extensions

• typing_extensions

• unittest-xml-reporting

• uritemplate

• us

• uuid6

• wfdb

• wsproto

• zlib

• zope.index

Added the following Python BuiltIn libraries with Direct status:

• aifc

• array

• ast

• asynchat

• asyncio

• asyncore

• atexit

• audioop

• base64

• bdb

• binascii

• bitsect

• builtins

• bz2

• calendar

• cgi

• cgitb

• chunk

• cmath

• cmd

• code

• codecs

• codeop

• colorsys

• compileall

• concurrent

• contextlib

• contextvars

• copy

• copyreg

• cprofile

• crypt

• csv

• ctypes

• curses

• dbm

• difflib

• dis

• distutils

• doctest

• email

• ensurepip

• enum

• errno

• faulthandler

• fcntl

• filecmp

• fileinput

• fnmatch

• fractions

• ftplib

• functools

• gc

• getopt

• getpass

• gettext

• graphlib

• grp

• gzip

• hashlib

• heapq

• hmac

• html

• http

• idlelib

• imaplib

• imghdr

• imp

• importlib

• inspect

• ipaddress

• itertools

• keyword

• linecache

• locale

• lzma

• mailbox

• mailcap

• marshal

• math

• mimetypes

• mmap

• modulefinder

• msilib

• multiprocessing

• netrc

• nis

• nntplib

• numbers

• operator

• optparse

• ossaudiodev

• pdb

• pickle

• pickletools

• pipes

• pkgutil

• platform

• plistlib

• poplib

• posix

• pprint

• profile

• pstats

• pty

• pwd

• py_compile

• pyclbr

• pydoc

• queue

• quopri

• random

• re

• reprlib

• resource

• rlcompleter

• runpy

• sched

• secrets

• select

• selectors

• shelve

• shlex

• signal

• site

• sitecustomize

• smtpd

• smtplib

• sndhdr

• socket

• socketserver

• spwd

• sqlite3

• ssl

• stat

• string

• stringprep

• struct

• subprocess

• sunau

• symtable

• sysconfig

• syslog

• tabnanny

• tarfile

• telnetlib

• tempfile

• termios

• test

• textwrap

• threading

• timeit

• tkinter

• token

• tokenize

• tomllib

• trace

• traceback

• tracemalloc

• tty

• turtle

• turtledemo

• types

• unicodedata

• urllib

• uu

• uuid

• venv

• warnings

• wave

• weakref

• webbrowser

• wsgiref

• xdrlib

• xml

• xmlrpc

• zipapp

• zipfile

• zipimport

• zoneinfo

Added the following Python BuiltIn libraries with NotSupported status:

• msvcrt

• winreg

• winsound

Modifications¶

• Mise à jour de .NET version sur v9.0.0.

• Amélioration de l’EWI SPRKPY1068.

• Mise à jour de la version de l’API Snowpark Python prise en charge par l’outil SMA de 1.24.0 à 1.25.0.

• Mise à jour du modèle de rapport détaillé, avec désormais la version Snowpark pour Pandas.

• Modification des bibliothèques suivantes de BuiltIn en ThirdPartyLib.

  • configparser

  • dataclasses

  • pathlib

  • readline

  • statistics

  • zlib

Updated the mapping status for the following Pandas elements, from Direct to Partial:

• pandas.core.frame.DataFrame.add

• pandas.core.frame.DataFrame.aggregate

• pandas.core.frame.DataFrame.all

• pandas.core.frame.DataFrame.apply

• pandas.core.frame.DataFrame.astype

• pandas.core.frame.DataFrame.cumsum

• pandas.core.frame.DataFrame.div

• pandas.core.frame.DataFrame.dropna

• pandas.core.frame.DataFrame.eq

• pandas.core.frame.DataFrame.ffill

• pandas.core.frame.DataFrame.fillna

• pandas.core.frame.DataFrame.floordiv

• pandas.core.frame.DataFrame.ge

• pandas.core.frame.DataFrame.groupby

• pandas.core.frame.DataFrame.gt

• pandas.core.frame.DataFrame.idxmax

• pandas.core.frame.DataFrame.idxmin

• pandas.core.frame.DataFrame.inf

• pandas.core.frame.DataFrame.join

• pandas.core.frame.DataFrame.le

• pandas.core.frame.DataFrame.loc

• pandas.core.frame.DataFrame.lt

• pandas.core.frame.DataFrame.mask

• pandas.core.frame.DataFrame.merge

• pandas.core.frame.DataFrame.mod

• pandas.core.frame.DataFrame.mul

• pandas.core.frame.DataFrame.ne

• pandas.core.frame.DataFrame.nunique

• pandas.core.frame.DataFrame.pivot_table

• pandas.core.frame.DataFrame.pow

• pandas.core.frame.DataFrame.radd

• pandas.core.frame.DataFrame.rank

• pandas.core.frame.DataFrame.rdiv

• pandas.core.frame.DataFrame.rename

• pandas.core.frame.DataFrame.replace

• pandas.core.frame.DataFrame.resample

• pandas.core.frame.DataFrame.rfloordiv

• pandas.core.frame.DataFrame.rmod

• pandas.core.frame.DataFrame.rmul

• pandas.core.frame.DataFrame.rolling

• pandas.core.frame.DataFrame.round

• pandas.core.frame.DataFrame.rpow

• pandas.core.frame.DataFrame.rsub

• pandas.core.frame.DataFrame.rtruediv

• pandas.core.frame.DataFrame.shift

• pandas.core.frame.DataFrame.skew

• pandas.core.frame.DataFrame.sort_index

• pandas.core.frame.DataFrame.sort_values

• pandas.core.frame.DataFrame.sub

• pandas.core.frame.DataFrame.to_dict

• pandas.core.frame.DataFrame.transform

• pandas.core.frame.DataFrame.transpose

• pandas.core.frame.DataFrame.truediv

• pandas.core.frame.DataFrame.var

• pandas.core.indexes.datetimes.date_range

• pandas.core.reshape.concat.concat

• pandas.core.reshape.melt.melt

• pandas.core.reshape.merge.merge

• pandas.core.reshape.pivot.pivot_table

• pandas.core.reshape.tile.cut

• pandas.core.series.Series.add

• pandas.core.series.Series.aggregate

• pandas.core.series.Series.all

• pandas.core.series.Series.any

• pandas.core.series.Series.cumsum

• pandas.core.series.Series.div

• pandas.core.series.Series.dropna

• pandas.core.series.Series.eq

• pandas.core.series.Series.ffill

• pandas.core.series.Series.fillna

• pandas.core.series.Series.floordiv

• pandas.core.series.Series.ge

• pandas.core.series.Series.gt

• pandas.core.series.Series.lt

• pandas.core.series.Series.mask

• pandas.core.series.Series.mod

• pandas.core.series.Series.mul

• pandas.core.series.Series.multiply

• pandas.core.series.Series.ne

• pandas.core.series.Series.pow

• pandas.core.series.Series.quantile

• pandas.core.series.Series.radd

• pandas.core.series.Series.rank

• pandas.core.series.Series.rdiv

• pandas.core.series.Series.rename

• pandas.core.series.Series.replace

• pandas.core.series.Series.resample

• pandas.core.series.Series.rfloordiv

• pandas.core.series.Series.rmod

• pandas.core.series.Series.rmul

• pandas.core.series.Series.rolling

• pandas.core.series.Series.rpow

• pandas.core.series.Series.rsub

• pandas.core.series.Series.rtruediv

• pandas.core.series.Series.sample

• pandas.core.series.Series.shift

• pandas.core.series.Series.skew

• pandas.core.series.Series.sort_index

• pandas.core.series.Series.sort_values

• pandas.core.series.Series.std

• pandas.core.series.Series.sub

• pandas.core.series.Series.subtract

• pandas.core.series.Series.truediv

• pandas.core.series.Series.value_counts

• pandas.core.series.Series.var

• pandas.core.series.Series.where

• pandas.core.tools.numeric.to_numeric

Updated the mapping status for the following Pandas elements, from NotSupported to Direct:

• pandas.core.frame.DataFrame.attrs

• pandas.core.indexes.base.Index.to_numpy

• pandas.core.series.Series.str.len

• pandas.io.html.read_html

• pandas.io.xml.read_xml

• pandas.core.indexes.datetimes.DatetimeIndex.mean

• pandas.core.resample.Resampler.indices

• pandas.core.resample.Resampler.nunique

• pandas.core.series.Series.items

• pandas.core.tools.datetimes.to_datetime

• pandas.io.sas.sasreader.read_sas

• pandas.core.frame.DataFrame.attrs

• pandas.core.frame.DataFrame.style

• pandas.core.frame.DataFrame.items

• pandas.core.groupby.generic.DataFrameGroupBy.head

• pandas.core.groupby.generic.DataFrameGroupBy.median

• pandas.core.groupby.generic.DataFrameGroupBy.min

• pandas.core.groupby.generic.DataFrameGroupBy.nunique

• pandas.core.groupby.generic.DataFrameGroupBy.tail

• pandas.core.indexes.base.Index.is_boolean

• pandas.core.indexes.base.Index.is_floating

• pandas.core.indexes.base.Index.is_integer

• pandas.core.indexes.base.Index.is_monotonic_decreasing

• pandas.core.indexes.base.Index.is_monotonic_increasing

• pandas.core.indexes.base.Index.is_numeric

• pandas.core.indexes.base.Index.is_object

• pandas.core.indexes.base.Index.max

• pandas.core.indexes.base.Index.min

• pandas.core.indexes.base.Index.name

• pandas.core.indexes.base.Index.names

• pandas.core.indexes.base.Index.rename

• pandas.core.indexes.base.Index.set_names

• pandas.core.indexes.datetimes.DatetimeIndex.day_name

• pandas.core.indexes.datetimes.DatetimeIndex.month_name

• pandas.core.indexes.datetimes.DatetimeIndex.time

• pandas.core.indexes.timedeltas.TimedeltaIndex.ceil

• pandas.core.indexes.timedeltas.TimedeltaIndex.days

• pandas.core.indexes.timedeltas.TimedeltaIndex.floor

• pandas.core.indexes.timedeltas.TimedeltaIndex.microseconds

• pandas.core.indexes.timedeltas.TimedeltaIndex.nanoseconds

• pandas.core.indexes.timedeltas.TimedeltaIndex.round

• pandas.core.indexes.timedeltas.TimedeltaIndex.seconds

• pandas.core.reshape.pivot.crosstab

• pandas.core.series.Series.dt.round

• pandas.core.series.Series.dt.time

• pandas.core.series.Series.dt.weekday

• pandas.core.series.Series.is_monotonic_decreasing

• pandas.core.series.Series.is_monotonic_increasing

Updated the mapping status for the following Pandas elements, from NotSupported to Partial:

• pandas.core.frame.DataFrame.align

• pandas.core.series.Series.align

• pandas.core.frame.DataFrame.tz_convert

• pandas.core.frame.DataFrame.tz_localize

• pandas.core.groupby.generic.DataFrameGroupBy.fillna

• pandas.core.groupby.generic.SeriesGroupBy.fillna

• pandas.core.indexes.datetimes.bdate_range

• pandas.core.indexes.datetimes.DatetimeIndex.std

• pandas.core.indexes.timedeltas.TimedeltaIndex.mean

• pandas.core.resample.Resampler.asfreq

• pandas.core.resample.Resampler.quantile

• pandas.core.series.Series.map

• pandas.core.series.Series.tz_convert

• pandas.core.series.Series.tz_localize

• pandas.core.window.expanding.Expanding.count

• pandas.core.window.rolling.Rolling.count

• pandas.core.groupby.generic.DataFrameGroupBy.aggregate

• pandas.core.groupby.generic.SeriesGroupBy.aggregate

• pandas.core.frame.DataFrame.applymap

• pandas.core.series.Series.apply

• pandas.core.groupby.generic.DataFrameGroupBy.bfill

• pandas.core.groupby.generic.DataFrameGroupBy.ffill

• pandas.core.groupby.generic.SeriesGroupBy.bfill

• pandas.core.groupby.generic.SeriesGroupBy.ffill

• pandas.core.frame.DataFrame.backfill

• pandas.core.frame.DataFrame.bfill

• pandas.core.frame.DataFrame.compare

• pandas.core.frame.DataFrame.unstack

• pandas.core.frame.DataFrame.asfreq

• pandas.core.series.Series.backfill

• pandas.core.series.Series.bfill

• pandas.core.series.Series.compare

• pandas.core.series.Series.unstack

• pandas.core.series.Series.asfreq

• pandas.core.series.Series.argmax

• pandas.core.series.Series.argmin

• pandas.core.indexes.accessors.CombinedDatetimelikeProperties.microsecond

• pandas.core.indexes.accessors.CombinedDatetimelikeProperties.nanosecond

• pandas.core.indexes.accessors.CombinedDatetimelikeProperties.day_name

• pandas.core.indexes.accessors.CombinedDatetimelikeProperties.month_name

• pandas.core.indexes.accessors.CombinedDatetimelikeProperties.month_start

• pandas.core.indexes.accessors.CombinedDatetimelikeProperties.month_end

• pandas.core.indexes.accessors.CombinedDatetimelikeProperties.is_year_start

• pandas.core.indexes.accessors.CombinedDatetimelikeProperties.is_year_end

• pandas.core.indexes.accessors.CombinedDatetimelikeProperties.is_quarter_start

• pandas.core.indexes.accessors.CombinedDatetimelikeProperties.is_quarter_end

• pandas.core.indexes.accessors.CombinedDatetimelikeProperties.is_leap_year

• pandas.core.indexes.accessors.CombinedDatetimelikeProperties.floor

• pandas.core.indexes.accessors.CombinedDatetimelikeProperties.ceil

• pandas.core.groupby.generic.DataFrameGroupBy.idxmax

• pandas.core.groupby.generic.DataFrameGroupBy.idxmin

• pandas.core.groupby.generic.DataFrameGroupBy.std

• pandas.core.indexes.timedeltas.TimedeltaIndex.mean

• pandas.core.tools.timedeltas.to_timedelta

Problème connu¶

• Cette version inclut un problème lors de la conversion de l’exemple de projet qui ne fonctionnera pas sur cette version. Il sera corrigé dans la prochaine version.

Application de bureau¶

• Ajout d’un lien vers le guide de dépannage dans la fenêtre modale du rapport de panne.

Versions de base de SMA incluses¶

• Snowpark Conversion Core 4.15.0

Ajouté¶

• Added the following PySpark elements to ConversionStatusPySpark.csv file as NotSupported:

  • pyspark.sql.streaming.readwriter.DataStreamReader.table

  • pyspark.sql.streaming.readwriter.DataStreamReader.schema

  • pyspark.sql.streaming.readwriter.DataStreamReader.options

  • pyspark.sql.streaming.readwriter.DataStreamReader.option

  • pyspark.sql.streaming.readwriter.DataStreamReader.load

  • pyspark.sql.streaming.readwriter.DataStreamReader.format

  • pyspark.sql.streaming.query.StreamingQuery.awaitTermination

  • pyspark.sql.streaming.readwriter.DataStreamWriter.partitionBy

  • pyspark.sql.streaming.readwriter.DataStreamWriter.toTable

  • pyspark.sql.streaming.readwriter.DataStreamWriter.trigger

  • pyspark.sql.streaming.readwriter.DataStreamWriter.queryName

  • pyspark.sql.streaming.readwriter.DataStreamWriter.outputMode

  • pyspark.sql.streaming.readwriter.DataStreamWriter.format

  • pyspark.sql.streaming.readwriter.DataStreamWriter.option

  • pyspark.sql.streaming.readwriter.DataStreamWriter.foreachBatch

  • pyspark.sql.streaming.readwriter.DataStreamWriter.start

Modifications¶

• Mise à jour du format des EWIs Hive SQL.

  • SPRKHVSQL1001

  • SPRKHVSQL1002

  • SPRKHVSQL1003

  • SPRKHVSQL1004

  • SPRKHVSQL1005

  • SPRKHVSQL1006

• Mise à jour du format des EWIs Spark SQL .

  • SPRKSPSQL1001

  • SPRKSPSQL1002

  • SPRKSPSQL1003

  • SPRKSPSQL1004

  • SPRKSPSQL1005

  • SPRKSPSQL1006

Correction¶

• Correction d’un bogue qui faisait que certains éléments PySpark n’étaient pas identifiés par l’outil.

• Correction de l’erreur de correspondance entre le nombre d’appels ThirdParty identifiés et le nombre d’appels ThirdParty importés.

Versions de base de SMA incluses¶

• Snowpark Conversion Core 4.14.0

Ajout¶

• Ajout des éléments Spark suivants à ConversionStatusPySpark. csv :

  • pyspark.broadcast.Broadcast.value

  • pyspark.conf.SparkConf.getAll

  • pyspark.conf.SparkConf.setAll

  • pyspark.conf.SparkConf.setMaster

  • pyspark.context.SparkContext.addFile

  • pyspark.context.SparkContext.addPyFile

  • pyspark.context.SparkContext.binaryFiles

  • pyspark.context.SparkContext.setSystemProperty

  • pyspark.context.SparkContext.version

  • pyspark.files.SparkFiles

  • pyspark.files.SparkFiles.get

  • pyspark.rdd.RDD.count

  • pyspark.rdd.RDD.distinct

  • pyspark.rdd.RDD.reduceByKey

  • pyspark.rdd.RDD.saveAsTextFile

  • pyspark.rdd.RDD.take

  • pyspark.rdd.RDD.zipWithIndex

  • pyspark.sql.context.SQLContext.udf

  • pyspark.sql.types.StructType.simpleString

Modifications¶

• Updated the documentation of the Pandas EWIs, PNDSPY1001, PNDSPY1002 and PNDSPY1003 SPRKSCL1137 to align with a standardized format, ensuring consistency and clarity across all the EWIs.

• Updated the documentation of the following Scala EWIs: SPRKSCL1106 and SPRKSCL1107. To be aligned with a standardized format, ensuring consistency and clarity across all the EWIs.

Correction¶

• Correction du bogue qui provoquait l’affichage des symboles UserDefined dans l’inventaire des utilisations tierces.

Versions de base de SMA incluses¶

• Snowpark Conversion Core 4.13.1

Interface de ligne de commande¶

Modifications

• Ajout d’un horodatage au dossier de sortie.

Ajouté¶

• Ajout d’une colonne « Langue source » dans la table de mappages de la bibliothèque

• Added Others as a new category in the Pandas API Summary table of the DetailedReport.docx

Modifications¶

• Updated the documentation for Python EWI SPRKPY1058.

• Updated the message for the pandas EWI PNDSPY1002 to show the relate pandas element.

• Mise à jour de la façon dont nous créons les rapports .csv, qui sont maintenant écrasés après une deuxième exécution.

Correction¶

• Correction d’un bogue qui faisait que les fichiers notebooks n’étaient pas générés dans la sortie.

• Fixed the replacer for get and set methods from pyspark.sql.conf.RuntimeConfig, the replacer now match the correct full names.

• Correction de la version incorrecte de la balise de requête.

• Correction des paquets UserDefined signalés comme ThirdPartyLib.

Versions de base de SMA incluses¶

• Snowpark Conversion Core 4.12.0

Application de bureau¶

Correction

• Correction des problèmes de casse dans les options –sql.

Supprimé

• Supprimez le nom de la plateforme dans le message show-ac.

Ajouté¶

• Ajout de la prise en charge de Snowpark Python 1.23.0 et 1.24.0.

• Added a new EWI for the pyspark.sql.dataframe.DataFrame.writeTo function. All the usages of this function will now have the EWI SPRKPY1087.

Modifications¶

• Updated the documentation of the Scala EWIs from SPRKSCL1137 to SPRKSCL1156 to align with a standardized format, ensuring consistency and clarity across all the EWIs.

• Updated the documentation of the Scala EWIs from SPRKSCL1117 to SPRKSCL1136 to align with a standardized format, ensuring consistency and clarity across all the EWIs.

• Mise à jour du message qui s’affiche pour les EWIs suivants :

  • SPRKPY1082

  • SPRKPY1083

• Updated the documentation of the Scala EWIs from SPRKSCL1100 to SPRKSCL1105, from SPRKSCL1108 to SPRKSCL1116; from SPRKSCL1157 to SPRKSCL1175; to align with a standardized format, ensuring consistency and clarity across all the EWIs.

• Mise à jour du statut de mappage des éléments PySpark suivants de NotSupported à Direct avec EWI :

  • pyspark.sql.readwriter.DataFrameWriter.option => snowflake.snowpark.DataFrameWriter.option: All the usages of this function now have the EWI SPRKPY1088

  • pyspark.sql.readwriter.DataFrameWriter.options => snowflake.snowpark.DataFrameWriter.options: All the usages of this function now have the EWI SPRKPY1089

• Mise à jour du statut de mappage des éléments PySpark suivants de Contournement en Renommer :

  • pyspark.sql.readwriter.DataFrameWriter.partitionBy => snowflake.snowpark.DataFrameWriter.partition_by

• Mise à jour de la documentation de l’EWI : SPRKSCL1000, SPRKSCL1001, SPRKSCL1002, SPRKSCL1100, SPRKSCL1101, SPRKSCL1102, SPRKSCL1103, SPRKSCL1104, SPRKSCL1105.

Supprimée¶

• Removed the pyspark.sql.dataframe.DataFrameStatFunctions.writeTo element from the conversion status, this element does not exist.

Obsolète¶

• Les codes EWI suivants sont obsolètes :

  • SPRKPY1081

  • SPRKPY1084

Ajouté¶

• Added a new column called Url to the Issues.csv file, which redirects to the corresponding EWI documentation.

• Ajout d’EWIs pour les éléments Spark suivants :

  • [SPRKPY1082] pyspark.sql.readwriter.DataFrameReader.load

  • [SPRKPY1083] pyspark.sql.readwriter.DataFrameWriter.save

  • [SPRKPY1084] pyspark.sql.readwriter.DataFrameWriter.option

  • [SPRKPY1085] pyspark.ml.feature.VectorAssembler

  • [SPRKPY1086] pyspark.ml.linalg.VectorUDT

• Ajout de 38 nouveaux éléments Pandas :

  • pandas.core.frame.DataFrame.select

  • andas.core.frame.DataFrame.str

  • pandas.core.frame.DataFrame.str.replace

  • pandas.core.frame.DataFrame.str.upper

  • pandas.core.frame.DataFrame.to_list

  • pandas.core.frame.DataFrame.tolist

  • pandas.core.frame.DataFrame.unique

  • pandas.core.frame.DataFrame.values.tolist

  • pandas.core.frame.DataFrame.withColumn

  • pandas.core.groupby.generic._SeriesGroupByScalar

  • pandas.core.groupby.generic._SeriesGroupByScalar[S1].agg

  • pandas.core.groupby.generic._SeriesGroupByScalar[S1].aggregate

  • pandas.core.indexes.datetimes.DatetimeIndex.year

  • pandas.core.series.Series.columns

  • pandas.core.tools.datetimes.to_datetime.date

  • pandas.core.tools.datetimes.to_datetime.dt.strftime

  • pandas.core.tools.datetimes.to_datetime.strftime

  • pandas.io.parsers.readers.TextFileReader.apply

  • pandas.io.parsers.readers.TextFileReader.astype

  • pandas.io.parsers.readers.TextFileReader.columns

  • pandas.io.parsers.readers.TextFileReader.copy

  • pandas.io.parsers.readers.TextFileReader.drop

  • pandas.io.parsers.readers.TextFileReader.drop_duplicates

  • pandas.io.parsers.readers.TextFileReader.fillna

  • pandas.io.parsers.readers.TextFileReader.groupby

  • pandas.io.parsers.readers.TextFileReader.head

  • pandas.io.parsers.readers.TextFileReader.iloc

  • pandas.io.parsers.readers.TextFileReader.isin

  • pandas.io.parsers.readers.TextFileReader.iterrows

  • pandas.io.parsers.readers.TextFileReader.loc

  • pandas.io.parsers.readers.TextFileReader.merge

  • pandas.io.parsers.readers.TextFileReader.rename

  • pandas.io.parsers.readers.TextFileReader.shape

  • pandas.io.parsers.readers.TextFileReader.to_csv

  • pandas.io.parsers.readers.TextFileReader.to_excel

  • pandas.io.parsers.readers.TextFileReader.unique

  • pandas.io.parsers.readers.TextFileReader.values

  • pandas.tseries.offsets

Versions de base de SMA incluses¶

• Snowpark Conversion Core 4.10.0

Application de bureau¶

Correction¶

• Correction d’un bogue qui entraînait l’affichage par SMA de l’étiquette SnowConvert au lieu de Snowpark Migration Accelerator dans la barre de menu de la version Windows.

• Fixed a bug that caused the SMA to crash when it did not have read and write permissions to the .config directory in macOS and the AppData directory in Windows.

Interface de ligne de commande¶

Modifications

• Renamed the CLI executable name from snowct to sma.

• Suppression de l’argument de la langue source, de sorte que vous n’avez plus besoin de spécifier si vous exécutez une évaluation / conversion Python ou Scala.

• Les arguments de ligne de commande pris en charge par CLI ont été élargis par l’ajout des nouveaux arguments suivants :

  • --enableJupyter | -j: Flag to indicate if the conversion of Databricks notebooks to Jupyter is enabled or not.

  • --sql | -f: Database engine syntax to be used when a SQL command is detected.

  • --customerEmail | -e: Configure the customer email.

  • --customerCompany | -c: Configure the customer company.

  • --projectName | -p: Configure the customer project.

• Mise à jour de certains textes pour refléter le nom correct de l’application, afin d’assurer la cohérence et la clarté de tous les messages.

• Mise à jour des conditions d’utilisation de l’application.

• Mise à jour et développement de la documentation de CLI afin de refléter les dernières fonctions, améliorations et modifications.

• Mise à jour du texte qui s’affiche avant de procéder à l’exécution de l’outil SMA afin d’améliorer la qualité de l’information

• Mise à jour de CLI pour accepter « Oui » comme argument valide lors de la demande de confirmation de l’utilisateur.

• Allowed the CLI to continue the execution without waiting for user interaction by specifying the argument -y or --yes.

• Updated the help information of the --sql argument to show the values that this argument expects.

Ajouté¶

• Added a new EWI for the pyspark.sql.readwriter.DataFrameWriter.partitionBy function. All the usages of this function will now have the EWI SPRKPY1081.

• Added a new column called Technology to the ImportUsagesInventory.csv file.

Modifications¶

• Updated the Third-Party Libraries readiness score to also take into account the Unknown libraries.

• Updated the AssessmentFiles.zip file to include .json files instead of .pam files.

• Amélioration du mécanisme de conversion de CSV en JSON pour rendre le traitement des inventaires plus performant.

• Amélioration de la documentation des EWIs suivants :

  • SPRKPY1029

  • SPRKPY1054

  • SPRKPY1055

  • SPRKPY1063

  • SPRKPY1075

  • SPRKPY1076

• Updated the mapping status of the following Spark Scala elements from Direct to Rename.

  • org.apache.spark.sql.functions.shiftLeft => com.snowflake.snowpark.functions.shiftleft

  • org.apache.spark.sql.functions.shiftRight => com.snowflake.snowpark.functions.shiftright

• Updated the mapping status of the following Spark Scala elements from Not Supported to Direct.

  • org.apache.spark.sql.functions.shiftleft => com.snowflake.snowpark.functions.shiftleft

  • org.apache.spark.sql.functions.shiftright => com.snowflake.snowpark.functions.shiftright