
Sap Hana Ml
- 330 installs
- 399 repo stars
- Updated August 4, 2026
- secondsky/sap-skills
Build in-database ML with SAP HANA: PAL/APL algorithms, model training in SQLScript, scoring pipelines, and embedding predictions beside transactional data.
About
Covers SAP HANA machine learning with PAL/APL, SQLScript pipelines, in-database training, and embedded scoring next to transactional ERP data. Suited to SaaS and API solutions needing governed, low-latency predictions without exporting sensitive data to external ML platforms.
- SAP HANA ML (PAL/APL)
- In-database model training
- SQLScript scoring pipelines
- Embedded ERP predictions
- Low-latency analytics on HANA
Sap Hana Ml by the numbers
- 330 all-time installs (skills.sh)
- +29 installs in the week ending Aug 5, 2026 (Skillselion tracking)
- Ranked #563 of 2,064 Data Science & ML skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/secondsky/sap-skills --skill sap-hana-mlAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 330 |
|---|---|
| repo stars | ★ 399 |
| Last updated | August 4, 2026 |
| Repository | secondsky/sap-skills ↗ |
What it does
Build in-database ML with SAP HANA: PAL/APL algorithms, model training in SQLScript, scoring pipelines, and embedding predictions beside transactional data.
Files
SAP HANA ML Python Client (hana-ml)
Related Skills
- sap-dependency-security: Use for secure dependency pinning and upgrade workflows in Python/auxiliary tooling used alongside HANA ML stacks
When to Use This Skill
Use this skill when building machine learning workflows with the hana-ml Python client, using PAL/APL algorithms, querying HANA DataFrames, training or scoring models in-database, using AutoML, visualizing model output, or troubleshooting Python-to-HANA ML connections.
Common Issues
| Issue | First check |
|---|---|
| Connection fails | Verify HANA host, port, TLS/encryption, user privileges, and network allowlists. |
| PAL/APL algorithm missing | Confirm the HANA system has the required AFL/PAL/APL libraries installed and licensed. |
| DataFrame collection is slow | Push filtering/projection into HANA and avoid collecting large frames into Python. |
Package Version: 2.22.241011 Last Verified: 2025-11-27
Table of Contents
---
Installation & Setup
pip install hana-mlRequirements: Python 3.8+, SAP HANA 2.0 SPS03+ or SAP HANA Cloud
---
Quick Start
Connection & DataFrame
from hana_ml import ConnectionContext
# Connect
conn = ConnectionContext(
address='<hostname>',
port=443,
user='<username>',
password='<password>',
encrypt=True
)
# Create DataFrame
df = conn.table('MY_TABLE', schema='MY_SCHEMA')
print(f"Shape: {df.shape}")
df.head(10).collect()PAL Classification
from hana_ml.algorithms.pal.unified_classification import UnifiedClassification
# Train model
clf = UnifiedClassification(func='RandomDecisionTree')
clf.fit(train_df, features=['F1', 'F2', 'F3'], label='TARGET')
# Predict & evaluate
predictions = clf.predict(test_df, features=['F1', 'F2', 'F3'])
score = clf.score(test_df, features=['F1', 'F2', 'F3'], label='TARGET')APL AutoML
from hana_ml.algorithms.apl.classification import AutoClassifier
# Automated classification
auto_clf = AutoClassifier()
auto_clf.fit(train_df, label='TARGET')
predictions = auto_clf.predict(test_df)Model Persistence
from hana_ml.model_storage import ModelStorage
ms = ModelStorage(conn)
clf.name = 'MY_CLASSIFIER'
ms.save_model(model=clf, if_exists='replace')---
Core Libraries
PAL (Predictive Analysis Library)
- 100+ algorithms executed in-database
- Categories: Classification, Regression, Clustering, Time Series, Preprocessing
- Key classes:
UnifiedClassification,UnifiedRegression,KMeans,ARIMA - See:
references/PAL_ALGORITHMS.mdfor complete list
APL (Automated Predictive Library)
- AutoML capabilities with automatic feature engineering
- Key classes:
AutoClassifier,AutoRegressor,GradientBoostingClassifier - See:
references/APL_ALGORITHMS.mdfor details
DataFrames
- Lazy evaluation - builds SQL until
collect()called - In-database processing for optimal performance
- See:
references/DATAFRAME_REFERENCE.mdfor complete API
Visualizers
- EDA plots, model explanations, metrics
- SHAP integration for model interpretability
- See:
references/VISUALIZERS.mdfor 14 visualization modules
---
Common Patterns
Train-Test Split
from hana_ml.algorithms.pal.partition import train_test_val_split
train, test, val = train_test_val_split(
data=df,
training_percentage=0.7,
testing_percentage=0.2,
validation_percentage=0.1
)Feature Importance
# APL models
importance = auto_clf.get_feature_importances()
# PAL models
from hana_ml.algorithms.pal.preprocessing import FeatureSelection
fs = FeatureSelection()
fs.fit(train_df, features=features, label='TARGET')Pipeline
from hana_ml.algorithms.pal.pipeline import Pipeline
from hana_ml.algorithms.pal.preprocessing import Imputer, FeatureNormalizer
pipeline = Pipeline([
('imputer', Imputer(strategy='mean')),
('normalizer', FeatureNormalizer()),
('classifier', UnifiedClassification(func='RandomDecisionTree'))
])---
Best Practices
1. Use lazy evaluation - Operations build SQL without execution until collect() 2. Leverage in-database processing - Keep data in HANA for performance 3. Use Unified interfaces - Consistent APIs across algorithms 4. Save models - Use ModelStorage for persistence 5. Explain predictions - Use SHAP explainers for interpretability 6. Monitor AutoML - Use PipelineProgressStatusMonitor for long-running jobs
---
Bundled Resources
Reference Files
- `references/DATAFRAME_REFERENCE.md` (479 lines)
- ConnectionContext API, DataFrame operations, SQL generation
- `references/PAL_ALGORITHMS.md` (869 lines)
- Complete PAL algorithm reference (100+ algorithms)
- Classification, Regression, Clustering, Time Series, Preprocessing
- `references/APL_ALGORITHMS.md` (534 lines)
- AutoML capabilities, automated feature engineering
- AutoClassifier, AutoRegressor, GradientBoosting classes
- `references/VISUALIZERS.md` (704 lines)
- 14 visualization modules (EDA, SHAP, metrics, time series)
- Plot types, configuration, export options
- `references/SUPPORTING_MODULES.md` (626 lines)
- Model storage, spatial analytics, graph algorithms
- Text mining, statistics, error handling
---
Error Handling
from hana_ml.ml_exceptions import Error
try:
clf.fit(train_df, features=features, label='TARGET')
except Error as e:
print(f"HANA ML Error: {e}")---
Documentation
- Official Docs: https://help.sap.com/doc/1d0ebfe5e8dd44d09606814d83308d4b/2.0.07/en-US/hana_ml.html
- PyPI Package: https://pypi.org/project/hana-ml/
SAP HANA ML Skill
Claude Code skill for SAP HANA Machine Learning Python Client (hana-ml) development.
Overview
This skill provides comprehensive guidance for building machine learning solutions using SAP HANA's in-database ML capabilities with Python. It covers the hana-ml library including PAL (Predictive Analysis Library), APL (Automated Predictive Library), DataFrames, visualizations, and model management.
Version
- Skill Version: 1.1.0
- hana-ml Version: 2.22.241011
- Last Verified: 2025-11-27
Capability Index
| Capability | Status |
|---|---|
| Commands | 1: /hana-ml-experiment-plan |
| Agents | 0 |
| Hooks | No |
| MCP | No |
| LSP | No |
| Source Freshness | last_verified: 2025-11-27; package/source freshness noted in third-pass audit. |
| Verification | npm run validate; HANA connection, PAL, and APL checks pending. |
Auto-Trigger Keywords
This skill activates when working with:
Library & Connection
- hana-ml, hana_ml, hana ml
- SAP HANA machine learning, HANA ML
- ConnectionContext, HANA connection
- hdbcli, SAP HANA Python driver
DataFrame Operations
- HANA DataFrame, hana_ml.dataframe
- create_dataframe_from_pandas
- collect(), filter(), select()
- HANA table operations
PAL Algorithms
- PAL, Predictive Analysis Library
- UnifiedClassification, UnifiedRegression, UnifiedClustering
- KMeans, DBSCAN, clustering HANA
- LogisticRegression HANA, DecisionTree HANA
- ARIMA HANA, AutoARIMA, time series HANA
- LSTM HANA, GRUAttention
- HybridGradientBoostingClassifier, HybridGradientBoostingRegressor
- FeatureNormalizer, PCA HANA, Imputer HANA
- SMOTE HANA, train_test_val_split
- GridSearchCV HANA, RandomSearchCV HANA
APL Algorithms
- APL, Automated Predictive Library
- AutoClassifier, AutoRegressor
- GradientBoostingClassifier APL
- AutoTimeSeries, HANA forecasting
- AutoML HANA, automated machine learning HANA
Visualizations
- EDAVisualizer, HANA visualization
- ShapleyExplainer, SHAP HANA
- TreeModelDebriefing
- MetricsVisualizer, confusion matrix HANA
- plot_acf, plot_pacf, seasonal_plot
Model Management
- ModelStorage, save_model HANA
- load_model HANA, model persistence
- export_apply_code
Advanced Features
- GeometryDBSCAN, spatial clustering HANA
- LatentDirichletAllocation, topic modeling HANA
- Pipeline HANA ML
- feature_importances HANA
Statistics & Testing
- ttest HANA, chi_squared HANA
- f_oneway, ANOVA HANA
- distribution_fit, KDE HANA
- kaplan_meier HANA, survival analysis
Spatial & Graph
- hana_ml.spatial, spatial analytics
- hana_ml.graph, graph algorithms
- PageRank HANA, LinkPrediction
- create_dataframe_from_shapefile
Scheduling & Artifacts
- schedule_fit, schedule_predict
- hana_ml.artifacts, model artifacts
- get_artifacts_recorder
Error Keywords
- hana_ml.ml_exceptions
- ConnectionContext error
- PAL algorithm error
- HANA ML fit error
Contents
sap-hana-ml/
├── SKILL.md # Main skill file
├── README.md # This file
└── references/
├── DATAFRAME_REFERENCE.md # Complete DataFrame API
├── PAL_ALGORITHMS.md # All PAL algorithms (100+)
├── APL_ALGORITHMS.md # All APL algorithms (AutoML)
├── VISUALIZERS.md # Visualization API (14 submodules)
└── SUPPORTING_MODULES.md # Model storage, spatial, graph, statsQuick Start
from hana_ml import ConnectionContext
from hana_ml.algorithms.pal.unified_classification import UnifiedClassification
# Connect to HANA
conn = ConnectionContext(address='host', port=443, user='user', password='pwd', encrypt=True)
# Load data
df = conn.table('TRAINING_DATA')
# Train model
clf = UnifiedClassification(func='RandomDecisionTree')
clf.fit(df, features=['F1', 'F2'], label='TARGET')
# Predict
predictions = clf.predict(conn.table('TEST_DATA'), features=['F1', 'F2'])Use Cases
- Building classification models with PAL or APL
- Creating regression models for prediction
- Clustering analysis with KMeans, DBSCAN
- Time series forecasting with ARIMA, LSTM
- AutoML with APL AutoClassifier/AutoRegressor
- Model explainability with SHAP
- Feature engineering and preprocessing
- Hyperparameter tuning with GridSearchCV
- Model persistence and deployment
Documentation Links
- Main Documentation
- Installation Guide
- DataFrame API
- PAL Algorithms
- APL Algorithms
- Visualizers
- PyPI Package
License
GPL-3.0
APL (Automated Predictive Library) Algorithms Reference
Module: hana_ml.algorithms.apl Documentation: https://help.sap.com/doc/1d0ebfe5e8dd44d09606814d83308d4b/2.0.07/en-US/hana_ml.algorithms.apl.html
---
Overview
APL (Automated Predictive Library) provides AutoML capabilities with:
- Automatic feature engineering
- Automatic algorithm selection
- Built-in model optimization
- Explainability features
---
Classification
AutoClassifier
Automated classification with automatic feature selection and algorithm optimization.
from hana_ml.algorithms.apl.classification import AutoClassifier
auto_clf = AutoClassifier(
variable_auto_selection=True,
variable_selection_best_iteration=True,
cutting_strategy='maximize_predictive_power', # or 'maximize_f1_score'
)
# Train
auto_clf.fit(
train_df,
label='TARGET',
key='ID' # Optional: row identifier
)
# Check training status
print(auto_clf.is_fitted())
# Predict
predictions = auto_clf.predict(test_df)
# Score
score = auto_clf.score(test_df, label='TARGET')GradientBoostingClassifier
Multi-class gradient boosting implementation.
from hana_ml.algorithms.apl.gradient_boosting_classification import GradientBoostingClassifier
gbc = GradientBoostingClassifier(
early_stopping_patience=10,
eval_metric='MultiClassLogLoss',
learning_rate=0.1,
max_depth=6,
max_iterations=100
)
gbc.fit(train_df, label='TARGET')
predictions = gbc.predict(test_df)GradientBoostingBinaryClassifier
Specialized binary classification with gradient boosting.
from hana_ml.algorithms.apl.gradient_boosting_classification import GradientBoostingBinaryClassifier
gbc_binary = GradientBoostingBinaryClassifier(
early_stopping_patience=10,
learning_rate=0.1,
max_depth=6
)
gbc_binary.fit(train_df, label='IS_POSITIVE')
predictions = gbc_binary.predict(test_df)
probabilities = gbc_binary.predict_proba(test_df)---
Regression
AutoRegressor
Automated regression with built-in feature engineering.
from hana_ml.algorithms.apl.regression import AutoRegressor
auto_reg = AutoRegressor(
variable_auto_selection=True,
polynomial_degree=1
)
auto_reg.fit(train_df, label='PRICE')
predictions = auto_reg.predict(test_df)
# Get performance metrics
metrics = auto_reg.get_performance_metrics()
print(metrics.collect())GradientBoostingRegressor
Gradient boosting for continuous target variables.
from hana_ml.algorithms.apl.gradient_boosting_regression import GradientBoostingRegressor
gbr = GradientBoostingRegressor(
early_stopping_patience=10,
eval_metric='RMSE',
learning_rate=0.1,
max_depth=6,
max_iterations=100
)
gbr.fit(train_df, label='PRICE')
predictions = gbr.predict(test_df)---
Time Series
AutoTimeSeries
Comprehensive time series forecasting with automatic model selection and parameter tuning.
from hana_ml.algorithms.apl.time_series import AutoTimeSeries
ts_model = AutoTimeSeries(
horizon=12, # Forecast horizon
last_training_time_point='2023-12-31', # Last training date
forecast_method='Default', # 'Default', 'ExponentialSmoothing', 'LinearRegression'
with_exogenous=True # Include exogenous variables
)
# Train with exogenous variables
ts_model.fit(
ts_df,
endog='SALES',
exog=['PROMOTION', 'HOLIDAY', 'PRICE']
)
# Forecast
forecast = ts_model.predict(horizon=12)
# Forecast with future exogenous values
forecast = ts_model.predict(
horizon=12,
exog_pred=future_exog_df
)
# Get accuracy metrics
accuracy = ts_model.get_accuracy_metrics()---
Clustering
AutoUnsupervisedClustering
Automatic clustering without target labels.
from hana_ml.algorithms.apl.clustering import AutoUnsupervisedClustering
auto_cluster = AutoUnsupervisedClustering(
max_clusters=10
)
auto_cluster.fit(data_df)
labels = auto_cluster.predict(data_df)
# Get cluster statistics
stats = auto_cluster.get_cluster_statistics()AutoSupervisedClustering
Clustering with labeled data guidance.
from hana_ml.algorithms.apl.clustering import AutoSupervisedClustering
sup_cluster = AutoSupervisedClustering()
sup_cluster.fit(data_df, label='SEGMENT')
predictions = sup_cluster.predict(new_data_df)---
Common Methods
All APL classes share these methods:
Training & Prediction
# Train model
model.fit(train_df, label='TARGET')
# Make predictions
predictions = model.predict(test_df)
# Calculate score
score = model.score(test_df, label='TARGET')
# Combined fit and predict
predictions = model.fit_predict(train_df, label='TARGET')Model State
# Check if model is trained
model.is_fitted()
# Get model summary
summary = model.get_summary()
print(summary.collect())
# Get detailed debrief report
debrief = model.get_debrief_report()
print(debrief.collect())Performance Analysis
# Performance metrics
metrics = model.get_performance_metrics()
print(metrics.collect())
# Feature importances
importance = model.get_feature_importances()
print(importance.collect())
# Feature contributions (for individual predictions)
contributions = model.get_feature_contributions(test_df)Model Persistence
# Save model to HANA table
model.name = 'MY_APL_MODEL'
model.save_model(model_table='APL_MODELS', if_exists='replace')
# Load model from HANA table
model.load_model(model_table='APL_MODELS')
# Save artifacts
model.save_artifact(artifact_table='APL_ARTIFACTS')Code Export
# Export model as apply code (for deployment)
apply_code = model.export_apply_code()
print(apply_code)
# Export for different runtimes
js_code = model.export_apply_code(target='JavaScript')
sql_code = model.export_apply_code(target='SQL')Reporting
# Build HTML report
model.build_report()
# Generate downloadable HTML report
html_content = model.generate_html_report()
# Display in notebook
model.generate_notebook_iframe_report()Distributed Processing
# Enable scale-out for large datasets
model.set_scale_out(enabled=True)
# Schedule asynchronous training
job_id = model.schedule_fit(train_df, label='TARGET')
# Schedule asynchronous prediction
job_id = model.schedule_predict(test_df)Explainability
# Add SHAP explainer to prediction phase
model.set_shapley_explainer_of_predict_phase(enabled=True)
# Get SHAP values for predictions
predictions_with_shap = model.predict(test_df)
# Access explainer
from hana_ml.visualizers.shap import ShapleyExplainer
explainer = ShapleyExplainer(model)
explainer.summary_plot(test_df)
explainer.force_plot(test_df.head(1))---
Parameters Reference
AutoClassifier Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
variable_auto_selection | bool | True | Automatic feature selection |
variable_selection_best_iteration | bool | True | Use best iteration for selection |
cutting_strategy | str | 'maximize_predictive_power' | Target optimization strategy |
polynomial_degree | int | 1 | Polynomial feature degree |
interactions_max_kept | int | None | Max interaction features |
AutoRegressor Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
variable_auto_selection | bool | True | Automatic feature selection |
polynomial_degree | int | 1 | Polynomial feature degree |
variable_selection_min_nb_of_final_variables | int | None | Min features to keep |
GradientBoosting Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
max_iterations | int | 100 | Maximum boosting iterations |
max_depth | int | 6 | Maximum tree depth |
learning_rate | float | 0.1 | Learning rate |
early_stopping_patience | int | 10 | Iterations without improvement |
eval_metric | str | varies | Evaluation metric |
subsample_ratio | float | 1.0 | Row sampling ratio |
colsample_ratio | float | 1.0 | Column sampling ratio |
AutoTimeSeries Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
horizon | int | None | Forecast horizon |
last_training_time_point | str | None | Last training timestamp |
forecast_method | str | 'Default' | Forecasting method |
with_exogenous | bool | False | Include exogenous variables |
season | int | None | Seasonal period |
with_decomposition | bool | True | Enable decomposition |
---
Model Storage with ModelStorage
from hana_ml.model_storage import ModelStorage
# Initialize storage
ms = ModelStorage(conn)
# Save APL model
auto_clf.name = 'CUSTOMER_CHURN_MODEL'
ms.save_model(
model=auto_clf,
if_exists='replace',
version=1
)
# List saved models
models = ms.list_models()
print(models)
# Load model
loaded_model = ms.load_model('CUSTOMER_CHURN_MODEL')
# Load specific version
loaded_model = ms.load_model('CUSTOMER_CHURN_MODEL', version=1)
# Delete model
ms.delete_model('CUSTOMER_CHURN_MODEL')
# Delete specific version
ms.delete_model('CUSTOMER_CHURN_MODEL', version=1)---
Visualization Integration
Model Debriefing
from hana_ml.visualizers.model_debriefing import TreeModelDebriefing
# For tree-based APL models
debriefing = TreeModelDebriefing(model)
# Tree visualization
debriefing.tree_debrief()
# Export tree
debriefing.tree_export(filename='model_tree.png')
# With DOT format
debriefing.tree_debrief_with_dot()SHAP Explainer
from hana_ml.visualizers.shap import ShapleyExplainer
explainer = ShapleyExplainer(auto_clf)
# Summary plot
explainer.summary_plot(test_df)
# Force plot for single prediction
explainer.force_plot(test_df.head(1))
# Beeswarm plot
explainer.get_beeswarm_plot_item(test_df)
# Dependence plot
explainer.get_dependence_plot_items(test_df, feature='AGE')
# Bar plot (feature importance)
explainer.get_bar_plot_item(test_df)Performance Metrics
from hana_ml.visualizers.metrics import MetricsVisualizer
mv = MetricsVisualizer()
# Confusion matrix
predictions = auto_clf.predict(test_df)
mv.plot_confusion_matrix(
y_true=test_df.select('TARGET').collect(),
y_pred=predictions.select('PREDICTED').collect()
)---
Best Practices
1. Feature Engineering
APL handles feature engineering automatically, but you can guide it:
auto_clf = AutoClassifier(
polynomial_degree=2, # Create polynomial features
interactions_max_kept=50 # Limit interaction terms
)2. Model Selection
Let APL optimize, but monitor performance:
# After training
metrics = auto_clf.get_performance_metrics()
importance = auto_clf.get_feature_importances()
# Review and adjust if needed
if importance.collect()['IMPORTANCE'].max() < 0.1:
# Features may not be predictive enough
pass3. Production Deployment
# Export apply code for deployment
apply_code = auto_clf.export_apply_code()
# Or use ModelStorage for HANA-native deployment
ms = ModelStorage(conn)
auto_clf.name = 'PRODUCTION_MODEL'
ms.save_model(model=auto_clf, version=1)
# Load in production
prod_model = ms.load_model('PRODUCTION_MODEL', version=1)4. Monitoring
Track model performance over time:
# Create model card
from hana_ml.algorithms.pal.model_selection import create_model_card
card = create_model_card(
model=auto_clf,
model_name='Customer Churn Predictor',
description='Predicts customer churn probability',
training_data_description='12 months of customer data',
intended_use='Marketing targeting'
)---
APL vs PAL Decision Guide
| Use Case | Recommendation |
|---|---|
| Quick prototyping | APL (automatic) |
| Production with custom requirements | PAL (granular control) |
| Feature engineering needed | APL (automatic) |
| Specific algorithm required | PAL (explicit selection) |
| Time series with complex seasonality | APL AutoTimeSeries |
| Ensemble methods | PAL (more options) |
| Explainability required | APL (built-in SHAP) |
| Deep learning | PAL (LSTM, MLP) |
SAP HANA DataFrame Reference
Module: hana_ml.dataframe Documentation: https://help.sap.com/doc/1d0ebfe5e8dd44d09606814d83308d4b/2.0.07/en-US/hana_ml.dataframe.html
Table of Contents
- ConnectionContext
- Constructor
- Connection Management Methods
- ABAP SQL Operations
- Schema Operations
- Table Operations
- View Operations
- Procedure Operations
- Temporary Table Management
- System Information
- Data Operations
- Streaming Operations
- SQL Execution
- DataFrame
- Creating DataFrames
- Properties
- Data Selection
- Filtering
- Data Transformation
- Sorting
- Aggregations
- Join Operations
- Set Operations
- Data Output
- Feature Engineering
- Data Quality
- Utility Methods
- Utility Functions
- Lazy Evaluation
---
ConnectionContext
Manages database connections to SAP HANA instances.
Constructor
from hana_ml import ConnectionContext
conn = ConnectionContext(
address='hostname', # HANA host
port=443, # Port (443 for HANA Cloud)
user='username', # Database user
password='password', # Password
encrypt=True, # SSL encryption (required for Cloud)
sslValidateCertificate=True, # Certificate validation
autocommit=True # Auto-commit transactions
)Connection Management Methods
| Method | Description |
|---|---|
close() | Terminate the database connection |
copy() | Duplicate connection settings |
get_connection_id() | Retrieve current connection identifier |
restart_session() | Reinitialize the database session |
cancel_session_process() | Stop running operations on current session |
ABAP SQL Operations
| Method | Description |
|---|---|
enable_abap_sql() | Enable ABAP SQL execution mode |
disable_abap_sql() | Disable ABAP SQL execution mode |
Schema Operations
| Method | Description |
|---|---|
create_schema(schema_name) | Create a new schema |
has_schema(schema_name) | Check if schema exists |
get_current_schema() | Get current default schema |
Table Operations
| Method | Description |
|---|---|
create_table(table_name, table_structure) | Create a new table |
drop_table(table_name) | Delete a table |
has_table(table_name) | Check if table exists |
get_tables(schema=None) | List all tables |
create_virtual_table(name, source) | Create virtual table reference |
table(name, schema=None) | Create DataFrame from table |
add_primary_key(table_name, columns) | Define primary keys on table |
View Operations
| Method | Description |
|---|---|
drop_view(view_name) | Delete a view |
Procedure Operations
| Method | Description |
|---|---|
get_procedures(schema=None) | List stored procedures |
drop_procedure(procedure_name) | Delete a stored procedure |
Temporary Table Management
| Method | Description |
|---|---|
get_temporary_tables() | List temporary tables |
clean_up_temporary_tables() | Remove all temporary tables |
System Information
| Method | Description |
|---|---|
hana_version() | Get full HANA version string |
hana_major_version() | Get major version number |
is_cloud_version() | Check if connected to HANA Cloud |
SQL Execution
| Method | Description |
|---|---|
sql(sql_string) | Execute SQL, return DataFrame |
execute_sql(sql_string) | Execute SQL without return |
explain_plan_statement(sql_string) | Get query execution plan |
Data Operations
| Method | Description |
|---|---|
copy_to_data_lake(df, target) | Export DataFrame to data lake |
to_sqlalchemy() | Get SQLAlchemy connection object |
Streaming Operations
| Method | Description |
|---|---|
upsert_streams_data(table, data) | Insert or update stream records |
update_streams_data(table, data) | Modify stream records |
---
DataFrame
Represents tabular data from HANA with lazy evaluation.
Properties
| Property | Description |
|---|---|
columns | List of column names |
shape | Tuple of (row_count, column_count) |
name | Underlying table/view name |
quoted_name | SQL-quoted table identifier |
description | Column metadata |
description_ext | Extended metadata |
geometries | Geometry column names |
srids | Spatial Reference IDs |
stats | Statistical summaries |
Data Selection
# Select specific columns
df.select('COL1', 'COL2', 'COL3')
# Deselect columns
df.deselect('UNWANTED_COL')
# Get first/last rows
df.head(10)
df.tail(10)
df.to_head() # Limit to first N rows
df.to_tail() # Limit to last N rowsFiltering
# Simple filter
df.filter("AGE > 30")
# Multiple conditions
df.filter("AGE > 30 AND SALARY > 50000")
# Check for value existence
df.has('COLUMN', 'value')
# Remove duplicates
df.distinct()
df.drop_duplicates()
# Handle missing values
df.dropna() # Remove rows with NULL
df.dropna(subset=['COL1']) # Only check specific columns
df.fillna(0) # Replace NULL with value
df.fillna({'COL1': 0, 'COL2': 'unknown'}) # Column-specificData Transformation
# Add columns
df.add_id('ROW_ID') # Add auto-increment ID
df.add_constant('NEW_COL', value=1) # Add constant column
# Rename
df.alias('new_df_name') # Rename DataFrame
df.rename_columns({'OLD': 'NEW'}) # Rename columns
# Type conversion
df.cast('COLUMN', 'DOUBLE') # Cast to specific type
df.auto_cast() # Auto-detect types
# Column operations
df.split_column('FULL_NAME', ' ', ['FIRST', 'LAST'])
df.concat_columns(['FIRST', 'LAST'], 'FULL_NAME', delimiter=' ')
# Value replacement
df.nullif('COLUMN', 'NA') # Replace value with NULL
df.replace('COLUMN', {'old': 'new'}) # Replace valuesSorting
df.sort('COLUMN', desc=True)
df.sort_values(['COL1', 'COL2'], ascending=[True, False])
df.sort_index()Aggregations
# Basic aggregations
df.count()
df.min('COLUMN')
df.max('COLUMN')
df.sum('COLUMN')
df.mean('COLUMN')
df.median('COLUMN')
df.stddev('COLUMN')
# Custom aggregations
df.agg([
('SALARY', 'mean', 'AVG_SALARY'),
('SALARY', 'max', 'MAX_SALARY'),
('AGE', 'count', 'COUNT')
])
# Correlation
df.corr('COL1', 'COL2')
# Value counts
df.value_counts('CATEGORY')
# Pivot table
df.pivot_table(
values='SALES',
index='REGION',
columns='QUARTER',
aggfunc='sum'
)
# Descriptive statistics
df.summary()
df.describe()Join Operations
Join conditions use SQL-style syntax with fully qualified column references:
# Inner join - condition uses SQL-like syntax with DataFrame names
df1.join(df2, condition='df1.ID = df2.ID') # ✓ Correct: explicit table refs
# Left join
df1.join(df2, condition='df1.ID = df2.ID', how='left')
# Right join
df1.join(df2, condition='df1.ID = df2.ID', how='right')
# Full outer join
df1.join(df2, condition='df1.ID = df2.ID', how='full')
# Join types: 'inner' (default), 'left', 'right', 'full'Important: Always use fully qualified column names (df1.COLUMN, df2.COLUMN) in join conditions to avoid ambiguity:
# ✓ Correct - explicit DataFrame references
df1.join(df2, condition='df1.ID = df2.CUSTOMER_ID', how='left')
# ✗ Avoid - ambiguous column references may cause errors
# df1.join(df2, condition='ID = CUSTOMER_ID', how='left')Set Operations
# Union
df1.union(df2)
# Generic set operations
df1.set_operations(df2, operation='EXCEPT') # Difference
df1.set_operations(df2, operation='INTERSECT') # IntersectionData Output
# Materialize to pandas
pdf = df.collect()
# Save to HANA table
df.save('NEW_TABLE', force=True)
df.save_nativedisktable('NATIVE_TABLE')
# Serialize
df.to_pickle('filename.pkl')
# Date conversion
df.to_datetime('DATE_COL', format='%Y-%m-%d')Feature Engineering
# Binning
df.bin('COLUMN', bins=[0, 25, 50, 75, 100], labels=['Q1', 'Q2', 'Q3', 'Q4'])
# Generate features
df.generate_feature('NEW_COL', 'COL1 + COL2')
# Computed columns
df.mutate('NEW_COL', 'COL1 * COL2')
# Difference calculation
df.diff('COLUMN', periods=1)Data Quality
# Check for empty
df.empty()
# Check for NULL values
df.hasna()
# Check for constant columns
df.has_constant_columns()
df.drop_constant_columns()
# Column type checking
df.is_numeric('COLUMN')Utility Methods
# Configuration
df.set_name('new_name')
df.set_index('ID_COLUMN')
df.set_source_table('ORIGINAL_TABLE')
# Column reordering
df.rearrange(['COL3', 'COL1', 'COL2'])
# Schema inspection
df.get_table_structure()
# Drop columns
df.drop('UNWANTED_COL')
# Validation
df.enable_validate_columns()
df.disable_validate_columns()
# Type definitions
df.generate_table_type()
df.declare_lttab_usage()---
Utility Functions
DataFrame Creation
from hana_ml.dataframe import (
create_dataframe_from_pandas,
create_dataframe_from_spark,
create_dataframe_from_shapefile
)
# From pandas
hdf = create_dataframe_from_pandas(
conn,
pandas_df,
table_name='MY_TABLE',
schema='MY_SCHEMA',
force=True # Overwrite if exists
)
# From Spark
hdf = create_dataframe_from_spark(conn, spark_df, table_name='MY_TABLE')
# From shapefile (geospatial)
hdf = create_dataframe_from_shapefile(conn, 'path/to/file.shp', table_name='GEO_TABLE')Data Import
from hana_ml.dataframe import import_csv_from
# Import CSV
hdf = import_csv_from(
conn,
'path/to/file.csv',
table_name='CSV_TABLE',
delimiter=',',
header=True
)Serialization
from hana_ml.dataframe import read_pickle
# Read pickled DataFrame
hdf = read_pickle(conn, 'filename.pkl')Reshaping
from hana_ml.dataframe import melt
# Unpivot DataFrame
melted = melt(
df,
id_vars=['ID', 'NAME'],
value_vars=['Q1', 'Q2', 'Q3', 'Q4'],
var_name='QUARTER',
value_name='SALES'
)SQL Utilities
from hana_ml.dataframe import quotename
# Safe SQL identifier quoting
safe_name = quotename('table-with-special-chars')---
Lazy Evaluation
HANA DataFrames use lazy evaluation - operations build SQL expressions without immediate execution.
# These operations build SQL but don't execute
filtered = df.filter("AGE > 30")
selected = filtered.select('NAME', 'AGE')
sorted_df = selected.sort('AGE', desc=True)
# Only collect() triggers execution
result = sorted_df.collect() # Returns pandas DataFrameBenefits:
- Query optimization across chained operations
- Reduced data transfer
- Push computations to database
PAL (Predictive Analysis Library) Algorithms Reference
Module: hana_ml.algorithms.pal Documentation: https://help.sap.com/doc/1d0ebfe5e8dd44d09606814d83308d4b/2.0.07/en-US/hana_ml.algorithms.pal.html
---
Unified Interfaces
Standardized APIs for common ML tasks.
UnifiedClassification
from hana_ml.algorithms.pal.unified_classification import UnifiedClassification
# Available functions
clf = UnifiedClassification(func='RandomDecisionTree') # or:
# 'DecisionTree', 'LogisticRegression', 'NaiveBayes',
# 'SVM', 'MLP', 'KNN', 'HybridGradientBoostingTree'
clf.fit(train_df, features=['F1', 'F2'], label='TARGET')
predictions = clf.predict(test_df, features=['F1', 'F2'])
score = clf.score(test_df, features=['F1', 'F2'], label='TARGET')UnifiedRegression
from hana_ml.algorithms.pal.unified_regression import UnifiedRegression
reg = UnifiedRegression(func='HybridGradientBoostingTree') # or:
# 'LinearRegression', 'DecisionTree', 'RandomDecisionTree',
# 'MLP', 'SVM', 'GLM', 'PolynomialRegression'
reg.fit(train_df, features=['F1', 'F2'], label='PRICE')
predictions = reg.predict(test_df, features=['F1', 'F2'])UnifiedClustering
from hana_ml.algorithms.pal.unified_clustering import UnifiedClustering
cluster = UnifiedClustering(func='KMeans', n_clusters=5)
cluster.fit(data_df, features=['F1', 'F2', 'F3'])
labels = cluster.predict(data_df, features=['F1', 'F2', 'F3'])---
AutoML
AutomaticClassification
from hana_ml.algorithms.pal.auto_ml import AutomaticClassification
auto_clf = AutomaticClassification(
generations=10,
population_size=20,
progress_indicator_id='PROGRESS_ID'
)
auto_clf.fit(train_df, features=features, label='TARGET')
best_model = auto_clf.best_pipeline_AutomaticRegression
from hana_ml.algorithms.pal.auto_ml import AutomaticRegression
auto_reg = AutomaticRegression(generations=10, population_size=20)
auto_reg.fit(train_df, features=features, label='PRICE')AutomaticTimeSeries
from hana_ml.algorithms.pal.auto_ml import AutomaticTimeSeries
auto_ts = AutomaticTimeSeries(generations=5)
auto_ts.fit(ts_df, endog='VALUE')
forecast = auto_ts.predict(forecast_length=30)Massive AutoML (Parallel Processing)
from hana_ml.algorithms.pal.auto_ml import (
MassiveAutomaticClassification,
MassiveAutomaticRegression,
MassiveAutomaticTimeSeries
)
# Process multiple models in parallel
massive_clf = MassiveAutomaticClassification()
massive_clf.fit(data_df, group_key='SEGMENT', features=features, label='TARGET')---
Classification Algorithms
Logistic Regression
from hana_ml.algorithms.pal.linear_model import LogisticRegression
lr = LogisticRegression(
max_iter=1000,
solver='newton', # 'newton', 'lbfgs', 'cyclical', 'stochastic'
multi_class='multinomial', # 'ovr', 'multinomial'
class_map0=None,
class_map1=None,
enet_alpha=1.0, # Elastic net mixing (0=L2, 1=L1)
lamb=0.0 # Regularization
)
lr.fit(train_df, features=['F1', 'F2'], label='TARGET')Decision Tree Classifier
from hana_ml.algorithms.pal.trees import DecisionTreeClassifier
dt = DecisionTreeClassifier(
algorithm='c45', # 'c45', 'chaid', 'cart'
max_depth=10,
min_samples_leaf=1,
min_records_of_parent=2,
min_records_of_leaf=1,
split_threshold=1e-5,
use_surrogate=False
)
dt.fit(train_df, features=features, label='TARGET')Random Decision Tree (Random Forest)
from hana_ml.algorithms.pal.trees import RDTClassifier
rdt = RDTClassifier(
n_estimators=100,
max_depth=None,
min_samples_leaf=1,
max_features='sqrt', # 'sqrt', 'log2', int, float
sample_fraction=1.0,
random_state=42
)
rdt.fit(train_df, features=features, label='TARGET')Hybrid Gradient Boosting Classifier
from hana_ml.algorithms.pal.trees import HybridGradientBoostingClassifier
hgb = HybridGradientBoostingClassifier(
n_estimators=100,
max_depth=6,
learning_rate=0.1,
split_threshold=0.01,
lamb=1.0, # L2 regularization
alpha=0.0 # L1 regularization
)
hgb.fit(train_df, features=features, label='TARGET')Support Vector Classification
from hana_ml.algorithms.pal.svm import SVC
svc = SVC(
kernel='rbf', # 'linear', 'poly', 'rbf', 'sigmoid'
c=1.0,
gamma='scale',
degree=3, # For poly kernel
shrinking=True
)
svc.fit(train_df, features=features, label='TARGET')Naive Bayes
from hana_ml.algorithms.pal.naive_bayes import NaiveBayes
nb = NaiveBayes(model_type='multinomial') # 'gaussian', 'multinomial'
nb.fit(train_df, features=features, label='TARGET')K-Nearest Neighbors
from hana_ml.algorithms.pal.neighbors import KNNClassifier
knn = KNNClassifier(
n_neighbors=5,
algorithm='brute-force', # 'brute-force', 'kd-tree'
metric='euclidean' # 'euclidean', 'manhattan', 'minkowski', 'chebyshev'
)
knn.fit(train_df, features=features, label='TARGET')MLP Classifier
from hana_ml.algorithms.pal.neural_network import MLPClassifier
mlp = MLPClassifier(
hidden_layer_sizes=(100, 50),
activation='relu', # 'relu', 'tanh', 'sigmoid'
output_activation='softmax',
learning_rate=0.001,
batch_size=32,
max_iter=200
)
mlp.fit(train_df, features=features, label='TARGET')One-Class SVM (Anomaly Detection)
from hana_ml.algorithms.pal.svm import OneClassSVM
ocsvm = OneClassSVM(kernel='rbf', nu=0.1, gamma='scale')
ocsvm.fit(train_df, features=features)
anomalies = ocsvm.predict(test_df, features=features)---
Regression Algorithms
Linear Regression
from hana_ml.algorithms.pal.linear_model import LinearRegression
lr = LinearRegression(
solver='qr', # 'qr', 'svd', 'cyclical', 'stochastic'
enet_alpha=1.0,
lamb=0.0,
intercept=True
)
lr.fit(train_df, features=features, label='PRICE')Polynomial Regression
from hana_ml.algorithms.pal.regression import PolynomialRegression
poly = PolynomialRegression(degree=2)
poly.fit(train_df, features=['X'], label='Y')Generalized Linear Model
from hana_ml.algorithms.pal.linear_model import GLM
glm = GLM(
family='gaussian', # 'gaussian', 'poisson', 'binomial', 'gamma', 'inverse_gaussian', 'negative_binomial'
link='identity', # 'identity', 'log', 'logit', 'probit', 'inverse', 'sqrt'
max_iter=100
)
glm.fit(train_df, features=features, label='Y')Support Vector Regression
from hana_ml.algorithms.pal.svm import SVR
svr = SVR(kernel='rbf', c=1.0, gamma='scale', epsilon=0.1)
svr.fit(train_df, features=features, label='PRICE')Decision Tree Regressor
from hana_ml.algorithms.pal.trees import DecisionTreeRegressor
dtr = DecisionTreeRegressor(
algorithm='cart',
max_depth=10,
min_samples_leaf=1
)
dtr.fit(train_df, features=features, label='PRICE')Random Forest Regressor
from hana_ml.algorithms.pal.trees import RDTRegressor
rdt = RDTRegressor(n_estimators=100, max_depth=10)
rdt.fit(train_df, features=features, label='PRICE')Hybrid Gradient Boosting Regressor
from hana_ml.algorithms.pal.trees import HybridGradientBoostingRegressor
hgb = HybridGradientBoostingRegressor(
n_estimators=100,
max_depth=6,
learning_rate=0.1
)
hgb.fit(train_df, features=features, label='PRICE')MLP Regressor
from hana_ml.algorithms.pal.neural_network import MLPRegressor
mlp = MLPRegressor(
hidden_layer_sizes=(100, 50),
activation='relu',
learning_rate=0.001,
max_iter=200
)
mlp.fit(train_df, features=features, label='PRICE')Cox Proportional Hazard Model
from hana_ml.algorithms.pal.regression import CoxProportionalHazardModel
cox = CoxProportionalHazardModel()
cox.fit(survival_df, features=features, label='TIME', event='EVENT')---
Clustering Algorithms
KMeans
from hana_ml.algorithms.pal.clustering import KMeans
kmeans = KMeans(
n_clusters=5,
init='first_k', # 'first_k', 'replace', 'no_replace', 'patent'
max_iter=100,
tol=1e-4,
distance_level='manhattan' # 'manhattan', 'euclidean', 'minkowski', 'chebyshev'
)
kmeans.fit(data_df, features=['F1', 'F2', 'F3'])
labels = kmeans.labels_
centers = kmeans.cluster_centers_KMedoids
from hana_ml.algorithms.pal.clustering import KMedoids
kmedoids = KMedoids(n_clusters=5, init='first_k', max_iter=100)
kmedoids.fit(data_df, features=features)DBSCAN
from hana_ml.algorithms.pal.clustering import DBSCAN
dbscan = DBSCAN(
eps=0.5,
minpts=5,
metric='euclidean' # 'euclidean', 'manhattan', 'minkowski'
)
dbscan.fit(data_df, features=features)
labels = dbscan.labels_Agglomerative Hierarchical Clustering
from hana_ml.algorithms.pal.clustering import AgglomerateHierarchicalClustering
ahc = AgglomerateHierarchicalClustering(
n_clusters=5,
affinity='euclidean',
linkage='average' # 'single', 'complete', 'average', 'ward', 'centroid', 'median'
)
ahc.fit(data_df, features=features)Spectral Clustering
from hana_ml.algorithms.pal.clustering import SpectralClustering
spectral = SpectralClustering(
n_clusters=5,
gamma=1.0,
n_components=None
)
spectral.fit(data_df, features=features)Gaussian Mixture
from hana_ml.algorithms.pal.mixture import GaussianMixture
gmm = GaussianMixture(
n_components=5,
covariance_type='full', # 'full', 'diag'
max_iter=100
)
gmm.fit(data_df, features=features)Self-Organizing Maps (SOM)
from hana_ml.algorithms.pal.clustering import SOM
som = SOM(
x_dim=10,
y_dim=10,
learning_rate=0.5,
max_iter=100
)
som.fit(data_df, features=features)GeometryDBSCAN (Spatial)
from hana_ml.algorithms.pal.clustering import GeometryDBSCAN
geo_dbscan = GeometryDBSCAN(eps=0.5, minpts=5)
geo_dbscan.fit(spatial_df, key='ID', features=['LOCATION'])---
Time Series Algorithms
ARIMA
from hana_ml.algorithms.pal.tsa.arima import ARIMA
arima = ARIMA(
order=(1, 1, 1), # (p, d, q)
seasonal_order=(1, 1, 1, 12), # (P, D, Q, s)
method='mle'
)
arima.fit(ts_df, endog='VALUE')
forecast = arima.predict(forecast_length=30)AutoARIMA
from hana_ml.algorithms.pal.tsa.auto_arima import AutoARIMA
auto_arima = AutoARIMA(
seasonal_period=12,
max_p=5, max_d=2, max_q=5,
max_P=2, max_D=1, max_Q=2
)
auto_arima.fit(ts_df, endog='VALUE')
forecast = auto_arima.predict(forecast_length=30)Exponential Smoothing
from hana_ml.algorithms.pal.tsa.exponential_smoothing import (
SingleExponentialSmoothing,
DoubleExponentialSmoothing,
TripleExponentialSmoothing,
AutoExponentialSmoothing
)
# Simple smoothing
ses = SingleExponentialSmoothing(alpha=0.3)
# Double (Holt's)
des = DoubleExponentialSmoothing(alpha=0.3, beta=0.1)
# Triple (Holt-Winters)
tes = TripleExponentialSmoothing(
alpha=0.3, beta=0.1, gamma=0.1,
seasonal='multiplicative',
seasonal_periods=12
)
# Automatic selection
auto_es = AutoExponentialSmoothing()
auto_es.fit(ts_df, endog='VALUE')LSTM
from hana_ml.algorithms.pal.tsa.lstm import LSTM
lstm = LSTM(
hidden_size=50,
num_layers=2,
learning_rate=0.001,
max_iter=100,
batch_size=32
)
lstm.fit(ts_df, endog='VALUE', exog=['FEATURE1', 'FEATURE2'])
forecast = lstm.predict(forecast_length=30, exog_pred=exog_future)BSTS (Bayesian Structural Time Series)
from hana_ml.algorithms.pal.tsa.bsts import BSTS
bsts = BSTS(
include_trend=True,
include_seasonal=True,
seasonal_period=12
)
bsts.fit(ts_df, endog='VALUE')Change Point Detection
from hana_ml.algorithms.pal.tsa.changepoint import CPD, BCPD
# Standard CPD
cpd = CPD(max_change_points=5)
cpd.fit(ts_df, endog='VALUE')
change_points = cpd.change_points_
# Bayesian CPD
bcpd = BCPD(max_change_points=5)
bcpd.fit(ts_df, endog='VALUE')---
Preprocessing
Feature Normalization
from hana_ml.algorithms.pal.preprocessing import FeatureNormalizer
normalizer = FeatureNormalizer(
method='min-max' # 'min-max', 'z-score', 'decimal'
)
normalizer.fit(train_df, features=features)
normalized = normalizer.transform(train_df, features=features)Imputation
from hana_ml.algorithms.pal.preprocessing import Imputer
imputer = Imputer(
strategy='mean' # 'mean', 'median', 'mode', 'delete'
)
imputer.fit(train_df, features=features)
imputed = imputer.transform(test_df, features=features)PCA
from hana_ml.algorithms.pal.decomposition import PCA
pca = PCA(n_components=10)
pca.fit(train_df, features=features)
transformed = pca.transform(test_df, features=features)
# Variance explained
print(pca.explained_variance_ratio_)Feature Selection
from hana_ml.algorithms.pal.preprocessing import FeatureSelection
fs = FeatureSelection(method='correlation')
fs.fit(train_df, features=features, label='TARGET')
selected_features = fs.selected_features_
importance = fs.importance_SMOTE (Oversampling)
from hana_ml.algorithms.pal.preprocessing import SMOTE
smote = SMOTE(k_neighbors=5, sampling_strategy='auto')
balanced_df = smote.fit_resample(imbalanced_df, features=features, label='TARGET')Train-Test Split
from hana_ml.algorithms.pal.partition import train_test_val_split
train, test, val = train_test_val_split(
data=df,
id_column='ID',
training_percentage=0.7,
testing_percentage=0.2,
validation_percentage=0.1,
random_seed=42
)---
Model Selection
Grid Search
from hana_ml.algorithms.pal.model_selection import GridSearchCV
param_grid = {
'n_estimators': [50, 100, 200],
'max_depth': [3, 5, 7],
'learning_rate': [0.01, 0.1]
}
grid_search = GridSearchCV(
estimator=HybridGradientBoostingClassifier(),
param_grid=param_grid,
cv=5,
scoring='accuracy'
)
grid_search.fit(train_df, features=features, label='TARGET')
print(grid_search.best_params_)
print(grid_search.best_score_)
best_model = grid_search.best_estimator_Random Search
from hana_ml.algorithms.pal.model_selection import RandomSearchCV
param_distributions = {
'n_estimators': [50, 100, 150, 200],
'max_depth': [3, 4, 5, 6, 7, 8],
'learning_rate': [0.01, 0.05, 0.1, 0.15, 0.2]
}
random_search = RandomSearchCV(
estimator=HybridGradientBoostingClassifier(),
param_distributions=param_distributions,
n_iter=20,
cv=5
)
random_search.fit(train_df, features=features, label='TARGET')Pipeline
from hana_ml.algorithms.pal.pipeline import Pipeline
pipeline = Pipeline([
('imputer', Imputer(strategy='mean')),
('normalizer', FeatureNormalizer(method='z-score')),
('pca', PCA(n_components=10)),
('classifier', HybridGradientBoostingClassifier())
])
pipeline.fit(train_df, features=features, label='TARGET')
predictions = pipeline.predict(test_df, features=features)---
Model Evaluation
Classification Metrics
from hana_ml.algorithms.pal.metrics import (
accuracy_score,
auc,
confusion_matrix,
multiclass_auc
)
accuracy = accuracy_score(y_true, y_pred)
auc_score = auc(y_true, y_proba)
cm = confusion_matrix(y_true, y_pred)Regression Metrics
from hana_ml.algorithms.pal.metrics import r2_score
r2 = r2_score(y_true, y_pred)---
Association Rules
Apriori
from hana_ml.algorithms.pal.association import Apriori
apriori = Apriori(
min_support=0.01,
min_confidence=0.5,
min_lift=1.0,
max_consequent=1,
max_item_length=5
)
apriori.fit(transaction_df, transaction='TRANS_ID', item='ITEM')
rules = apriori.result_FP-Growth
from hana_ml.algorithms.pal.association import FPGrowth
fpgrowth = FPGrowth(min_support=0.01, min_confidence=0.5)
fpgrowth.fit(transaction_df, transaction='TRANS_ID', item='ITEM')---
Recommender Systems
ALS (Alternating Least Squares)
from hana_ml.algorithms.pal.recommender import ALS
als = ALS(
n_factors=50,
max_iter=20,
regularization=0.1
)
als.fit(ratings_df, user='USER_ID', item='ITEM_ID', rating='RATING')
recommendations = als.recommend(user_id=1, n_items=10)Factorization Machines
from hana_ml.algorithms.pal.recommender import FFMClassifier, FFMRegressor
ffm = FFMClassifier(
n_factors=10,
max_iter=100,
learning_rate=0.1
)
ffm.fit(train_df, features=features, label='CLICK')---
Text Mining
Latent Dirichlet Allocation
from hana_ml.algorithms.pal.decomposition import LatentDirichletAllocation
lda = LatentDirichletAllocation(
n_components=10,
max_iter=100,
doc_topic_prior=None, # Alpha
topic_word_prior=None # Beta
)
lda.fit(text_df, features=['DOCUMENT'])
topics = lda.transform(text_df)CRF (Sequence Labeling)
from hana_ml.algorithms.pal.text import CRF
crf = CRF()
crf.fit(sequence_df, features=['TOKEN'], label='TAG')---
Statistics
Hypothesis Tests
from hana_ml.algorithms.pal.stats import (
ttest_1samp,
ttest_ind,
ttest_rel,
f_oneway,
chi2_contingency,
pearsonr_matrix
)
# One-sample t-test
t_stat, p_value = ttest_1samp(data_df, column='VALUE', popmean=0)
# Two-sample t-test
t_stat, p_value = ttest_ind(group1_df, group2_df, column='VALUE')
# ANOVA
f_stat, p_value = f_oneway(data_df, groups='GROUP', values='VALUE')
# Correlation matrix
corr = pearsonr_matrix(data_df, columns=['F1', 'F2', 'F3'])Distribution Functions
from hana_ml.algorithms.pal.stats import (
bernoulli, beta, binomial, chi_squared,
exponential, gamma, normal, poisson, uniform
)
# Generate random samples
samples = normal(conn, loc=0, scale=1, size=1000)---
Social Network Analysis
PageRank
from hana_ml.algorithms.pal.social_network import PageRank
pagerank = PageRank(damping=0.85, max_iter=100)
pagerank.fit(graph_df, source='FROM_NODE', target='TO_NODE')
ranks = pagerank.result_Link Prediction
from hana_ml.algorithms.pal.social_network import LinkPrediction
lp = LinkPrediction(method='common_neighbors')
lp.fit(graph_df, source='FROM_NODE', target='TO_NODE')
predictions = lp.predict(test_pairs_df)SAP HANA ML Supporting Modules Reference
Documentation: https://help.sap.com/doc/1d0ebfe5e8dd44d09606814d83308d4b/2.0.07/en-US/
---
Model Storage (hana_ml.model_storage)
Documentation: https://help.sap.com/doc/1d0ebfe5e8dd44d09606814d83308d4b/2.0.07/en-US/hana_ml.model_storage.html
ModelStorage Class
from hana_ml.model_storage import ModelStorage
# Initialize model storage
ms = ModelStorage(connection_context=conn)
# Save a trained model
model.name = 'MY_MODEL'
ms.save_model(
model=model,
if_exists='replace', # 'replace', 'error', 'append'
version=1
)
# List all saved models
models = ms.list_models()
print(models)
# Load a model
loaded_model = ms.load_model(name='MY_MODEL')
# Load specific version
loaded_model = ms.load_model(name='MY_MODEL', version=1)
# Delete a model
ms.delete_model(name='MY_MODEL')
# Delete specific version
ms.delete_model(name='MY_MODEL', version=1)Model Persistence in Algorithms
All PAL and APL algorithms support:
# Save model directly
model.save_model(model_table='MY_MODELS', if_exists='replace')
# Load model directly
model.load_model(model_table='MY_MODELS')---
Artifacts (hana_ml.artifacts)
Documentation: https://help.sap.com/doc/1d0ebfe5e8dd44d09606814d83308d4b/2.0.07/en-US/hana_ml.artifacts.html
Artifact Recording
# Get artifacts recorder from model
recorder = model.get_artifacts_recorder()
# Save model artifacts
model.save_artifact(artifact_table='MY_ARTIFACTS')Artifact Types
Artifacts include:
- Model weights and parameters
- Feature statistics
- Training metadata
- Performance metrics history
- Model configuration
---
Spatial (hana_ml.spatial)
Documentation: https://help.sap.com/doc/1d0ebfe5e8dd44d09606814d83308d4b/2.0.07/en-US/hana_ml.spatial.html
DataFrame Spatial Properties
# Access geometry columns
geometries = df.geometries
# Access spatial reference identifiers
srids = df.sridsCreating DataFrames from Shapefiles
from hana_ml.dataframe import create_dataframe_from_shapefile
# Import shapefile
geo_df = create_dataframe_from_shapefile(
connection_context=conn,
shp_file='path/to/file.shp',
table_name='GEO_DATA',
schema='MY_SCHEMA',
srid=4326 # WGS84
)GeometryDBSCAN (Spatial Clustering)
from hana_ml.algorithms.pal.clustering import GeometryDBSCAN
# Spatial density-based clustering
geo_dbscan = GeometryDBSCAN(
eps=0.5, # Maximum distance between points
minpts=5 # Minimum points for core point
)
geo_dbscan.fit(
data=spatial_df,
key='ID',
features=['LOCATION'] # Geometry column
)
# Get cluster labels
labels = geo_dbscan.labels_---
Graph (hana_ml.graph)
Documentation: https://help.sap.com/doc/1d0ebfe5e8dd44d09606814d83308d4b/2.0.07/en-US/hana_ml.graph.html
Graph Visualization (from hana_ml.visualizers.digraph)
from hana_ml.visualizers.digraph import (
Digraph,
MultiDigraph,
Node,
Edge,
DigraphConfig
)
# Configure graph layout
config = DigraphConfig()
config.set_digraph_layout('TB') # Top to Bottom
config.set_node_sep(1.5)
config.set_rank_sep(2.0)
config.set_text_layout('horizontal')
# Create directed graph
graph = Digraph(config=config)
# Add model nodes
graph.add_model_node(
node_id='preprocessing',
label='Preprocessing',
model_type='preprocessing'
)
graph.add_model_node(
node_id='model',
label='Classifier',
model_type='classification'
)
# Add Python node
graph.add_python_node(
node_id='custom',
label='Custom Logic'
)
# Add edges
graph.add_edge(source='preprocessing', target='model')
graph.add_edge(source='model', target='custom')
# Build and export
graph.build()
html = graph.generate_html()
graph.generate_notebook_iframe()
# Export to JSON
json_data = graph.to_json()MultiDigraph (Hierarchical Graphs)
from hana_ml.visualizers.digraph import MultiDigraph
# Create hierarchical graph
multi_graph = MultiDigraph()
# Add child digraphs for grouping
preprocessing_group = multi_graph.add_child_digraph(name='Preprocessing')
preprocessing_group.add_model_node('imputer', 'Imputer')
preprocessing_group.add_model_node('normalizer', 'Normalizer')
modeling_group = multi_graph.add_child_digraph(name='Modeling')
modeling_group.add_model_node('classifier', 'Classifier')
# Build and export
multi_graph.build()
multi_graph.generate_html()---
Graph Algorithms (hana_ml.graph.algorithms)
Documentation: https://help.sap.com/doc/1d0ebfe5e8dd44d09606814d83308d4b/2.0.07/en-US/hana_ml.graph.algorithms.html
Social Network Analysis (from PAL)
from hana_ml.algorithms.pal.social_network import PageRank, LinkPrediction
# PageRank
pagerank = PageRank(
damping=0.85,
max_iter=100,
tol=1e-6
)
pagerank.fit(
data=graph_df,
source='FROM_NODE',
target='TO_NODE',
weight='WEIGHT'
)
ranks = pagerank.result_
# Link Prediction
lp = LinkPrediction(
method='common_neighbors' # or 'jaccard', 'adamic_adar'
)
lp.fit(
data=graph_df,
source='FROM_NODE',
target='TO_NODE'
)
predictions = lp.predict(test_pairs_df)---
Text Mining (hana_ml.text.tm)
Documentation: https://help.sap.com/doc/1d0ebfe5e8dd44d09606814d83308d4b/2.0.07/en-US/hana_ml.text.tm.html
Latent Dirichlet Allocation (Topic Modeling)
from hana_ml.algorithms.pal.decomposition import LatentDirichletAllocation
# Topic modeling
lda = LatentDirichletAllocation(
n_components=10, # Number of topics
max_iter=100, # Maximum iterations
doc_topic_prior=None, # Alpha parameter
topic_word_prior=None, # Beta parameter
learning_method='batch', # 'batch' or 'online'
random_state=42
)
lda.fit(
data=text_df,
features=['DOCUMENT_TEXT']
)
# Get topic distributions for documents
topic_distributions = lda.transform(text_df)
# Get topic-word matrix
topic_words = lda.components_CRF (Conditional Random Fields)
from hana_ml.algorithms.pal.text import CRF
# Sequence labeling (NER, POS tagging)
crf = CRF()
crf.fit(
data=sequence_df,
features=['TOKEN', 'PREV_TOKEN', 'NEXT_TOKEN'],
label='TAG',
sequence_id='SENTENCE_ID'
)
# Predict labels
predictions = crf.predict(test_sequence_df, features=['TOKEN', 'PREV_TOKEN', 'NEXT_TOKEN'])WordCloud Visualization
from hana_ml.visualizers.word_cloud import WordCloud
# Create word cloud from text
wc = WordCloud(
width=800,
height=400,
background_color='white',
max_words=200,
min_font_size=10,
max_font_size=100
)
# Generate from text DataFrame
wc.generate_from_text(
data=text_df,
column='TEXT_CONTENT'
)
# Or from word frequencies
word_freq = {'python': 100, 'machine': 80, 'learning': 75}
wc.fit_words(word_freq)
# Generate from raw frequencies
wc.generate_from_frequencies(word_freq)
# Process text (tokenization, cleaning)
processed = wc.process_text('Raw text content here')
# Build word cloud
wc.build()
# Recolor
wc.recolor(colormap='viridis')
# Export
wc.to_file('wordcloud.png')
wc.to_svg('wordcloud.svg')
array = wc.to_array()---
Scheduler (hana_ml.hana_scheduler)
Documentation: https://help.sap.com/doc/1d0ebfe5e8dd44d09606814d83308d4b/2.0.07/en-US/hana_ml.hana_scheduler.html
Asynchronous Model Training
All APL and PAL models support scheduled execution:
# Schedule asynchronous training
job_id = model.schedule_fit(
data=train_df,
label='TARGET'
)
# Schedule asynchronous prediction
job_id = model.schedule_predict(
data=test_df
)
# Check operation logs
fit_log = model.get_fit_operation_log()
predict_log = model.get_predict_operation_log()---
Exceptions (hana_ml.ml_exceptions)
Documentation: https://help.sap.com/doc/1d0ebfe5e8dd44d09606814d83308d4b/2.0.07/en-US/hana_ml.ml_exceptions.html
Error Handling
from hana_ml.ml_exceptions import Error
try:
model.fit(train_df, label='TARGET')
except Error as e:
print(f"HANA ML Error: {e}")
# Handle specific error casesCommon Error Scenarios
| Error Type | Common Cause | Solution |
|---|---|---|
| Connection Error | Invalid credentials | Check connection parameters |
| Table Not Found | Schema/table mismatch | Verify table exists |
| Column Error | Invalid column name | Check DataFrame columns |
| Type Error | Data type mismatch | Cast columns appropriately |
| Memory Error | Large dataset | Use sampling or partitioning |
---
DocStore (hana_ml.docstore)
Documentation: https://help.sap.com/doc/1d0ebfe5e8dd44d09606814d83308d4b/2.0.07/en-US/hana_ml.docstore.html
Document Store Operations
The docstore module provides JSON document handling capabilities in SAP HANA.
# Note: Detailed API varies by version
# Typical operations include:
# - Create/manage collections
# - Insert/update/delete documents
# - Query JSON documents
# - Index management---
Statistics Functions (hana_ml.algorithms.pal.stats)
Distribution Functions
from hana_ml.algorithms.pal.stats import (
bernoulli, beta, binomial, cauchy, chi_squared,
exponential, gumbel, f, gamma, geometric,
lognormal, negative_binomial, normal, pert,
poisson, student_t, uniform, weibull,
multinomial, mcmc
)
# Generate random samples
samples = normal(conn, loc=0, scale=1, size=1000)
samples = uniform(conn, low=0, high=1, size=1000)
samples = poisson(conn, lam=5, size=1000)Statistical Tests
from hana_ml.algorithms.pal.stats import (
# T-tests
ttest_1samp, # One-sample t-test
ttest_ind, # Independent two-sample t-test
ttest_paired, # Paired t-test
# ANOVA
f_oneway, # One-way ANOVA
f_oneway_repeated, # Repeated measures ANOVA
# Chi-squared tests
chi_squared_goodness_of_fit,
chi_squared_independence,
# Non-parametric tests
wilcoxon, # Wilcoxon signed-rank test
median_test_1samp, # One-sample median test
grubbs_test, # Outlier detection
ks_test, # Kolmogorov-Smirnov test
# Correlation
pearsonr_matrix, # Pearson correlation matrix
covariance_matrix,
# Distribution
distribution_fit, # Fit distribution to data
KDE, # Kernel density estimation
# Other
univariate_analysis,
factor_analysis,
kaplan_meier_survival_analysis,
# Utility
entropy,
condition_index,
cdf,
ftest_equal_var,
quantile,
iqr, # Interquartile range
variance_test,
interval_quality,
benford_analysis # Benford's law analysis
)
# Example: One-sample t-test
t_stat, p_value = ttest_1samp(
data=df,
column='VALUE',
popmean=0
)
# Example: ANOVA
f_stat, p_value = f_oneway(
data=df,
groups='GROUP',
values='VALUE'
)
# Example: KDE
kde = KDE()
kde.fit(data_df, column='VALUE')
density = kde.evaluate(points_df)---
Time Series Utilities
from hana_ml.algorithms.pal.tsa import (
accuracy_measure, # Calculate forecast accuracy metrics
correlation, # Time series correlation
fft, # Fast Fourier Transform
dtw, # Dynamic Time Warping
fast_dtw # Fast DTW implementation
)
# Accuracy metrics
metrics = accuracy_measure(
actual=actual_df,
forecast=forecast_df,
measures=['MAE', 'MAPE', 'RMSE', 'SMAPE']
)
# Dynamic Time Warping
distance = dtw(
series1=ts1_df,
series2=ts2_df,
column='VALUE'
)
# Fast Fourier Transform
frequencies = fft(
data=ts_df,
column='VALUE'
)---
AutoML Configuration
from hana_ml.visualizers.automl_config import AutoMLConfig
# Configure AutoML parameters
config = AutoMLConfig()
# Get configuration dictionary
config_dict = config.get_config_dict()
# Generate HTML report of configuration
html = config.generate_html()---
Time Series Reports
from hana_ml.visualizers.time_series_report import (
TimeSeriesReport,
DatasetAnalysis
)
# Create time series analysis
analysis = DatasetAnalysis(
data=ts_df,
time_column='DATE',
value_column='VALUE'
)
# Available analysis items
pacf = analysis.pacf_item(lags=40)
ma = analysis.moving_average_item(window=7)
rolling = analysis.rolling_stddev_item(window=7)
seasonal = analysis.seasonal_item(period=12)
box = analysis.timeseries_box_item(groupby='month')
decompose = analysis.seasonal_decompose_items(period=12)
quarter = analysis.quarter_item()
outlier = analysis.outlier_item(method='iqr')
stationary = analysis.stationarity_item()
real = analysis.real_item()
change = analysis.change_points_item(max_points=5)
# Build comprehensive report
report = TimeSeriesReport()
report.addPage(title='Sales Analysis', items=[pacf, ma, seasonal])
report.addPages([
('Trend', [ma, rolling]),
('Seasonality', [seasonal, decompose])
])
report.build()
# Export
html = report.generate_html()
report.generate_notebook_iframe()
json_data = report.to_json()---
SHAP Time Series Explainer
from hana_ml.visualizers.shap import TimeSeriesExplainer
# Explain ARIMA model
arima_explainer = TimeSeriesExplainer(arima_model)
arima_explanation = arima_explainer.explain_arima_model(
data=ts_df,
time_column='DATE',
value_column='VALUE'
)
# Explain Additive Model
additive_explainer = TimeSeriesExplainer(additive_model)
additive_explanation = additive_explainer.explain_additive_model(
data=ts_df,
time_column='DATE',
value_column='VALUE'
)HANA ML Visualizers Reference
Module: hana_ml.visualizers Documentation: https://help.sap.com/doc/1d0ebfe5e8dd44d09606814d83308d4b/2.0.07/en-US/hana_ml.visualizers.html
---
EDA (Exploratory Data Analysis)
EDAVisualizer
from hana_ml.visualizers.eda import EDAVisualizer
import matplotlib.pyplot as plt
fig, ax = plt.subplots(figsize=(10, 6))
viz = EDAVisualizer(ax=ax)
# Distribution plot
viz.distribution_plot(
data=df,
column='AGE',
bins=20,
title='Age Distribution'
)
# Pie plot
viz.pie_plot(
data=df,
column='CATEGORY',
title='Category Distribution'
)
# Correlation plot
viz.correlation_plot(
data=df,
columns=['F1', 'F2', 'F3', 'F4'],
title='Feature Correlations'
)
# Scatter plot
viz.scatter_plot(
data=df,
x='AGE',
y='SALARY',
color='DEPARTMENT',
title='Age vs Salary'
)
# Bar plot
viz.bar_plot(
data=df,
column='CATEGORY',
aggregation='count',
title='Category Counts'
)
# Box plot
viz.box_plot(
data=df,
column='SALARY',
groupby='DEPARTMENT',
title='Salary by Department'
)
plt.show()Profiler
from hana_ml.visualizers.eda import Profiler
profiler = Profiler(conn)
profile = profiler.profile(df)
# Get statistics
print(profile.statistics)
print(profile.missing_values)
print(profile.unique_values)Time Series Plots
from hana_ml.visualizers.eda import (
plot_acf,
plot_pacf,
seasonal_plot,
quarter_plot,
timeseries_box_plot,
plot_change_points,
plot_moving_average,
plot_rolling_stddev,
plot_seasonal_decompose,
plot_time_series_outlier,
plot_psd
)
# Autocorrelation
plot_acf(ts_df, column='VALUE', lags=40)
# Partial autocorrelation
plot_pacf(ts_df, column='VALUE', lags=40)
# Seasonal plot
seasonal_plot(ts_df, column='VALUE', period=12)
# Quarter plot
quarter_plot(ts_df, column='VALUE')
# Box plot by time period
timeseries_box_plot(ts_df, column='VALUE', groupby='MONTH')
# Change point visualization
plot_change_points(ts_df, column='VALUE', change_points=[10, 50, 100])
# Moving average
plot_moving_average(ts_df, column='VALUE', window=7)
# Rolling standard deviation
plot_rolling_stddev(ts_df, column='VALUE', window=7)
# Seasonal decomposition
plot_seasonal_decompose(ts_df, column='VALUE', period=12)
# Outlier detection
plot_time_series_outlier(ts_df, column='VALUE', method='iqr')
# Power spectral density
plot_psd(ts_df, column='VALUE')Statistical Plots
from hana_ml.visualizers.eda import (
bubble_plot,
parallel_coordinates,
kdeplot,
hist
)
# Bubble plot
bubble_plot(
data=df,
x='AGE',
y='SALARY',
size='EXPERIENCE',
color='DEPARTMENT'
)
# Parallel coordinates
parallel_coordinates(
data=df,
columns=['F1', 'F2', 'F3', 'F4'],
color='CATEGORY'
)
# Kernel density estimation
kdeplot(data=df, column='VALUE')
# Histogram
hist(data=df, column='VALUE', bins=30)---
Metrics Visualization
MetricsVisualizer
from hana_ml.visualizers.metrics import MetricsVisualizer
import matplotlib.pyplot as plt
mv = MetricsVisualizer()
# Confusion matrix
mv.plot_confusion_matrix(
y_true=['A', 'B', 'A', 'C', 'B', 'A'],
y_pred=['A', 'B', 'B', 'C', 'B', 'A'],
labels=['A', 'B', 'C'],
title='Classification Results'
)
plt.show()---
Model Debriefing
TreeModelDebriefing
from hana_ml.visualizers.model_debriefing import TreeModelDebriefing
# Initialize with trained tree-based model
debriefing = TreeModelDebriefing(model)
# Interactive tree visualization
debriefing.tree_debrief()
# Export tree to file
debriefing.tree_export(filename='decision_tree.png', format='png')
# Parse tree structure
tree_structure = debriefing.tree_parse()
# Using DOT format
debriefing.tree_debrief_with_dot()
debriefing.tree_export_with_dot(filename='tree.dot')SHAP Explainer Integration
from hana_ml.visualizers.model_debriefing import TreeModelDebriefing
# Get SHAP values via debriefing
shap_values = debriefing.shapley_explainer(test_df)---
SHAP Visualization
ShapleyExplainer
from hana_ml.visualizers.shap import ShapleyExplainer
# Initialize with trained model
explainer = ShapleyExplainer(model)
# Summary plot (feature importance)
explainer.summary_plot(
data=test_df,
max_display=20, # Top N features
plot_type='bar' # or 'dot'
)
# Force plot (single prediction explanation)
explainer.force_plot(
data=test_df.head(1),
link='identity'
)
# Beeswarm plot
beeswarm = explainer.get_beeswarm_plot_item(test_df)
# Dependence plot
explainer.get_dependence_plot_items(
data=test_df,
feature='AGE',
interaction_feature='INCOME'
)
# Bar plot
bar_item = explainer.get_bar_plot_item(test_df)TimeSeriesExplainer
from hana_ml.visualizers.shap import TimeSeriesExplainer
ts_explainer = TimeSeriesExplainer(ts_model)
# Explain time series predictions
ts_explainer.summary_plot(ts_test_df)---
Dataset Reports
DatasetReportBuilder
from hana_ml.visualizers.dataset_report import DatasetReportBuilder
# Build comprehensive dataset report
report_builder = DatasetReportBuilder(conn)
# Build report
report = report_builder.build(
data=df,
columns=['AGE', 'SALARY', 'DEPARTMENT', 'TENURE'],
key='ID'
)
# Generate HTML report
html = report_builder.generate_html_report()
# Save to file
with open('dataset_report.html', 'w') as f:
f.write(html)
# Display in Jupyter notebook
report_builder.generate_notebook_iframe_report()---
Unified Reports
UnifiedReport
from hana_ml.visualizers.unified_report import UnifiedReport
# Build comprehensive model report
report = UnifiedReport(model)
# Build report content
report.build()
# Tree visualization (for tree-based models)
report.tree_debrief()
# Display in notebook
report.display()
# Get iframe for embedding
iframe = report.get_iframe_report()---
Time Series Reports
TimeSeriesReport
from hana_ml.visualizers.time_series_report import TimeSeriesReport, DatasetAnalysis
# Initialize report
ts_report = TimeSeriesReport()
# Add pages
ts_report.addPage(
title='Sales Analysis',
data=ts_df,
time_column='DATE',
value_column='SALES'
)
# Build report
ts_report.build()
# Generate HTML
html = ts_report.generate_html()DatasetAnalysis
from hana_ml.visualizers.time_series_report import DatasetAnalysis
analysis = DatasetAnalysis(
data=ts_df,
time_column='DATE',
value_column='VALUE'
)
# Various analysis methods
analysis.trend_analysis()
analysis.seasonality_analysis(period=12)
analysis.stationarity_test()---
Pipeline Visualization
Digraph
from hana_ml.visualizers.digraph import (
Digraph,
MultiDigraph,
Node,
Edge,
DigraphConfig
)
# Create pipeline visualization
config = DigraphConfig(
direction='TB', # Top to Bottom
node_style='rounded',
edge_style='solid'
)
graph = Digraph(config=config)
# Add nodes
graph.add_model_node(
node_id='preprocessing',
label='Data Preprocessing',
model_type='preprocessing'
)
graph.add_model_node(
node_id='classifier',
label='Random Forest',
model_type='classification'
)
# Add edges
graph.add_edge(
source='preprocessing',
target='classifier',
label='features'
)
# Build graph
graph.build()
# Generate HTML
html = graph.generate_html()
# Save to file
with open('pipeline.html', 'w') as f:
f.write(html)MultiDigraph
from hana_ml.visualizers.digraph import MultiDigraph
# Hierarchical graph
multi_graph = MultiDigraph()
# Add subgraphs
preprocessing_graph = multi_graph.add_subgraph('Preprocessing')
preprocessing_graph.add_model_node('imputer', 'Imputer')
preprocessing_graph.add_model_node('normalizer', 'Normalizer')
modeling_graph = multi_graph.add_subgraph('Modeling')
modeling_graph.add_model_node('model', 'Classifier')
# Connect subgraphs
multi_graph.add_edge('normalizer', 'model')
multi_graph.build()---
Word Cloud
WordCloud
from hana_ml.visualizers.word_cloud import WordCloud
# Initialize word cloud
wc = WordCloud(
width=800,
height=400,
background_color='white',
max_words=200
)
# Generate from text column
wc.generate_from_text(
data=text_df,
column='TEXT_CONTENT'
)
# Generate from word frequencies
word_freq = {'python': 100, 'machine': 80, 'learning': 75, 'data': 60}
wc.fit_words(word_freq)
# Process text
processed = wc.process_text('Raw text content here...')
# Save to file
wc.to_file('wordcloud.png')
# Save as SVG
wc.to_svg('wordcloud.svg')---
AutoML Progress Monitoring
PipelineProgressStatusMonitor
from hana_ml.visualizers.automl_progress import PipelineProgressStatusMonitor
# Monitor AutoML progress
monitor = PipelineProgressStatusMonitor(
connection_context=conn,
progress_id='MY_AUTOML_JOB'
)
# Start monitoring
monitor.start()
# Get current status
status = monitor.get_status()
print(status)
# Stop monitoring
monitor.stop()SimplePipelineProgressStatusMonitor
from hana_ml.visualizers.automl_progress import SimplePipelineProgressStatusMonitor
# Simple progress monitoring
simple_monitor = SimplePipelineProgressStatusMonitor(conn)
simple_monitor.monitor(auto_clf)---
AutoML Reports
BestPipelineReport
from hana_ml.visualizers.automl_report import BestPipelineReport
# Generate report for AutoML best pipeline
report = BestPipelineReport(auto_clf)
# Build report
report.build()
# Display
report.display()
# Get HTML
html = report.generate_html()---
M4 Sampling (Time Series Visualization)
m4_sampling
from hana_ml.visualizers.m4_sampling import (
m4_sampling,
get_min_index,
get_max_index
)
# M4 algorithm for efficient time series visualization
# Reduces data points while preserving visual patterns
sampled_df = m4_sampling(
data=large_ts_df,
column='VALUE',
num_pixels=1000 # Target number of points
)
# Get indices of extrema
min_idx = get_min_index(ts_df, column='VALUE')
max_idx = get_max_index(ts_df, column='VALUE')---
Forecast Visualization
forecast_line_plot
from hana_ml.visualizers.visualizer_base import forecast_line_plot
# Plot actual vs forecast
forecast_line_plot(
actual_data=actual_df,
forecast_data=forecast_df,
time_column='DATE',
actual_column='ACTUAL',
forecast_column='FORECAST',
confidence_lower='CI_LOWER',
confidence_upper='CI_UPPER',
title='Sales Forecast'
)---
Common Patterns
Saving Visualizations
import matplotlib.pyplot as plt
# Create figure - returns (Figure, Axes) tuple
fig, ax = plt.subplots(figsize=(12, 8))
# EDAVisualizer methods modify ax in-place and return ax for chaining
viz = EDAVisualizer(ax=ax)
ax = viz.distribution_plot(data=df, column='VALUE') # Returns matplotlib Axes
# Save to file - returns None, writes file to disk
plt.savefig('distribution.png', dpi=300, bbox_inches='tight')
# Close figure to free memory - important in loops/scripts
plt.close() # Returns None, releases figure resourcesMultiple Plots
import matplotlib.pyplot as plt
from hana_ml.visualizers.eda import EDAVisualizer
# Create 2x2 subplot grid
fig, axes = plt.subplots(2, 2, figsize=(14, 10)) # Returns (Figure, ndarray of Axes)
# Distribution - each plot method returns its Axes object
viz1 = EDAVisualizer(ax=axes[0, 0])
viz1.distribution_plot(data=df, column='AGE')
# Box plot
viz2 = EDAVisualizer(ax=axes[0, 1])
viz2.box_plot(data=df, column='SALARY', groupby='DEPARTMENT')
# Scatter
viz3 = EDAVisualizer(ax=axes[1, 0])
viz3.scatter_plot(data=df, x='AGE', y='SALARY')
# Correlation
viz4 = EDAVisualizer(ax=axes[1, 1])
viz4.correlation_plot(data=df, columns=['F1', 'F2', 'F3'])
# Adjust spacing between subplots - returns None, modifies figure state
plt.tight_layout()
# Save dashboard - returns None, writes file to disk
plt.savefig('eda_dashboard.png', dpi=300)
# Good practice: close figure to free memory (optional but recommended)
plt.close()Notebook Integration
# In Jupyter notebook
from IPython.display import HTML
# For HTML reports
html_content = report.generate_html()
HTML(html_content)
# For iframe reports
iframe = report.get_iframe_report()
display(iframe)---
Dependencies
The visualizers module requires direct and transitive dependencies:
Direct dependencies:
- matplotlib (static plots)
- plotly (interactive plots)
- graphviz (tree visualization)
- wordcloud (word cloud generation)
Transitive dependencies (required by above):
- numpy (required by matplotlib, plotting functions)
- pillow (required by wordcloud for image handling)
- pandas (DataFrame integration with visualizers)
# Install all visualization dependencies (explicit)
pip install matplotlib plotly graphviz wordcloud pillow numpy pandas
# Or install hana-ml which handles dependencies automatically
pip install hana-ml
# Note: hana-ml includes visualization dependencies by default
# Transitive dependencies are resolved automatically by pipNote: The graphviz Python package requires the Graphviz system binary to be installed separately:
# Ubuntu/Debian
apt-get install graphviz
# macOS
brew install graphviz
# Windows: Download from https://graphviz.org/download/