
Jcl Migration Analyzer
- 22 installs
- 14 repo stars
- Updated January 23, 2026
- dauquangthanh/hanoi-rainbow
jcl-migration-analyzer is a Hanoi Rainbow agent skill that parses JCL batch jobs so developers can migrate mainframe workflows to modern orchestration platforms.
About
The jcl-migration-analyzer skill analyzes legacy JCL and procedures to support migration to modern workflow orchestration such as Spring Batch, Apache Airflow, Kubernetes Jobs, or shell pipelines. It emphasizes dependency graphs, PROC parsing, GDG handling, and careful COND logic translation with supporting scripts for structure extraction. Use it when you have .jcl or .proc assets and need migration reports, complexity estimates, and implementation-ready orchestration strategies.
- Documents inverted COND semantics and IF/THEN/ELSE translation pitfalls
- Maps DD statements, GDG generations, and step-level data dependencies
- Includes analyze-dependencies and extract-structure automation scripts
- Targets Spring Batch, Airflow DAGs, Kubernetes Jobs, and Step Functions
Jcl Migration Analyzer by the numbers
- 22 all-time installs (skills.sh)
- Ranked #1,273 of 2,715 Automation & Workflows skills by installs in the Skillselion catalog
- Data as of Jul 29, 2026 (Skillselion catalog sync)
npx skills add https://github.com/dauquangthanh/hanoi-rainbow --skill jcl-migration-analyzerAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 22 |
|---|---|
| repo stars | ★ 14 |
| Last updated | January 23, 2026 |
| Repository | dauquangthanh/hanoi-rainbow ↗ |
How do you translate JCL steps, PROCs, and COND rules into reliable cloud or open-source batch workflows?
Parses JCL jobs and PROCs, maps DD dependencies and inverted COND logic, and plans Spring Batch, Airflow, or K8s job replacements.
Who is it for?
Developers modernizing mainframe batch jobs who can supply JCL and PROC files and need dependency-aware migration plans.
Skip if: Greenfield batch systems with no JCL sources or teams only tuning existing Airflow DAGs without legacy JCL input.
When should I use this skill?
Users mention JCL analysis, mainframe job migration, batch workflow conversion, COND logic, or .jcl files.
What you get
Structured job analysis JSON, dependency maps, conditional logic notes, and orchestration migration recommendations.
Files
JCL Migration Analyzer
Analyzes legacy JCL scripts for migration to modern batch processing and workflow orchestration systems like Spring Batch, Apache Airflow, Kubernetes Jobs, or shell scripts.
Overview
This skill provides comprehensive analysis and migration planning for JCL (Job Control Language) batch processing systems. It extracts job structures, converts JCL constructs to modern workflow patterns, maps data dependencies, and generates implementation-ready migration strategies.
Key Migration Focus: JCL to modern orchestration with proper handling of COND logic inversion, data dependencies (DD statements), GDG generations, procedures (PROCs), and batch workflow patterns.
When to Use This Skill
Use this skill when:
- Analyzing JCL job files (.jcl, .JCL) for modernization
- Planning migration from mainframe batch processing to modern workflow systems
- Converting JCL job steps to Spring Batch, Apache Airflow, or shell scripts
- Understanding JCL COND logic and conditional execution patterns
- Mapping JCL data sets (DD statements) to modern file operations
- Extracting JCL procedures (PROCs) and symbolic parameters
- Generating workflow definitions for orchestration platforms
- Estimating complexity and effort for JCL migration projects
- Creating migration documentation and strategy reports
- Modernizing mainframe batch jobs to cloud-native workflows
- User mentions: JCL analysis, mainframe job migration, batch workflow conversion, COND logic, job steps, procedures, workflow orchestration
Core Capabilities
1. Job Analysis
Extract job structure (JOB card), step sequences, program invocations (EXEC PGM/PROC), conditional logic (COND, IF/THEN/ELSE), return codes, data sets (DD statements), resource requirements, and symbolic parameters.
2. Data Dependency Mapping
Extract input/output datasets, temporary datasets, GDG handling, concatenation, DISP parameters, and data flow between steps.
3. Procedure Analysis
Parse PROC definitions, symbolic parameters, PROC overrides, nested procedures, INCLUDE statements, and JCLLIB references.
4. Workflow Migration
Generate Spring Batch jobs, Apache Airflow DAGs, Kubernetes Jobs, shell scripts, AWS Step Functions, or Azure Logic Apps.
5. Conditional Logic Translation
CRITICAL: COND logic is INVERTED! Map COND parameters, IF/THEN/ELSE, return codes, step bypassing, and restart logic to modern constructs.
Workflow
Step 1: Discover JCL Assets
Find JCL jobs and procedures in the workspace:
find . -name "*.jcl" -o -name "*.JCL"
find . -name "*.proc" -o -name "*.PROC"Use scripts/analyze-dependencies.sh or scripts/analyze-dependencies.ps1 to generate dependency graph in JSON format.
Step 2: Extract Structure
Use scripts/extract-structure.py to parse JCL files and extract:
- Job cards and parameters
- Step sequences and execution order
- Program/procedure invocations
- DD statements with DISP parameters
- COND and IF/THEN/ELSE logic
- Symbolic parameters
Output format: JSON with job structure, steps, and dependencies.
Step 3: Analyze Conditional Logic
CRITICAL: Identify and document COND logic (which is INVERTED):
COND=(0,NE)→ Run if previous RC ≠ 0 (run on ERROR)COND=(0,EQ)→ Skip if previous RC = 0 (skip on SUCCESS)- IF/THEN/ELSE uses normal logic (not inverted)
Create truth tables for complex conditional logic to avoid errors in migration.
Step 4: Map Data Dependencies
Track data flow between steps:
- Input datasets (DISP=SHR or OLD)
- Output datasets (DISP=NEW, CATLG)
- Temporary datasets (&&TEMP)
- GDG generations (GDG(0), GDG(+1))
- Dataset concatenations
Step 5: Estimate Complexity
Use scripts/estimate-complexity.py to calculate migration complexity based on:
- Number of job steps
- Conditional logic complexity (COND/IF/THEN/ELSE)
- Number of procedures (PROCs)
- Data dependency complexity
- Number of programs invoked
- GDG usage patterns
Step 6: Choose Target Platform
Select migration target based on requirements:
- Spring Batch: Java-based batch processing with comprehensive features
- Apache Airflow: Python-based workflow orchestration with rich UI
- Shell Scripts: Simple, lightweight for basic sequential processing
- Kubernetes Jobs: Container-based batch processing
- AWS Step Functions: Serverless workflow orchestration
- Azure Logic Apps: Cloud-based workflow integration
Step 7: Generate Migration Strategy
Create comprehensive migration report with:
1. Job Overview: Purpose, schedule, dependencies 2. Step Sequence: Detailed breakdown of each step 3. Data Flow Diagram: Input/output dependencies 4. Conditional Logic Map: COND translations (with inversion notes) 5. Target Implementation: Workflow definition in chosen platform 6. Migration Estimate: Effort, complexity score, risk assessment 7. Action Items: Prioritized tasks with acceptance criteria
Use template: assets/migration-report-template.md
Quick Reference
Critical: COND Logic is INVERTED
JCL COND (inverted):
//STEP020 EXEC PGM=PROG2,COND=(0,NE)Means: "Run if previous RC ≠ 0" → Run on ERROR!
Modern (normal logic):
if [ $rc -ne 0 ]; then run_prog2; fiJCL IF/THEN (normal logic):
//IF1 IF RC = 0 THEN
//STEP020 EXEC PGM=PROG2
//ENDIFModern:
if [ $rc -eq 0 ]; then run_prog2; fiCode Patterns
Simple Sequential:
//STEP010 EXEC PGM=PROG1
//INPUT DD DSN=INPUT.FILE,DISP=SHR
//OUTPUT DD DSN=OUTPUT.FILE,DISP=(NEW,CATLG)
//STEP020 EXEC PGM=PROG2
//INPUT DD DSN=OUTPUT.FILE,DISP=SHR#!/bin/bash
set -e
prog1 --input="input.file" --output="output.file" || exit 8
prog2 --input="output.file" || exit 8Conditional (COND - inverted!):
//STEP010 EXEC PGM=VALIDATE
//STEP020 EXEC PGM=PROCESS,COND=(0,NE)validate_data
rc=$?
if [ $rc -ne 0 ]; then process_data; fi # INVERTED!IF/THEN/ELSE (normal logic):
//STEP010 EXEC PGM=VALIDATE
//IF1 IF RC = 0 THEN
//STEP020 EXEC PGM=PROCESSOK
//ELSE
//STEP030 EXEC PGM=PROCESSERR
//ENDIFvalidate_data
rc=$?
if [ $rc -eq 0 ]; then processok; else processerr; fiProcedure:
//MYPROC PROC MEMBER=,INFILE=
//STEP1 EXEC PGM=PROG1
//SYSIN DD DSN=&MEMBER,DISP=SHR
// PENDfunction myproc() {
prog1 --sysin="$1" --input="$2"
}
myproc "test.data" "prod.file"Target Platforms
Spring Batch:
@Bean
public Job job() {
return jobBuilderFactory.get("job")
.start(step1()).next(step2())
.on("FAILED").to(errorStep())
.from(step2()).on("*").to(step3())
.end().build();
}Airflow DAG:
with DAG('job', schedule_interval='@daily') as dag:
step1 = BashOperator(task_id='step1', bash_command='prog1.sh')
step2 = BashOperator(task_id='step2', bash_command='prog2.sh')
step1 >> step2Key Patterns
Error Handling: COND-based → if [ $rc -ne 0 ]; then error_handler; fi GDG: GDG(0) → get_latest_generation, GDG(+1) → create_new_generation Concatenation: Multiple DD → cat file1 file2 file3 | process Restart: COND restart → checkpoint files (touch .checkpoint_step)
Return Code Reference
| RC | Meaning | Action |
|---|---|---|
| 0 | Success | Continue |
| 4 | Warning | Continue (informational) |
| 8 | Error | May continue based on COND |
| 12 | Severe Error | Typically stop |
| 16 | Fatal Error | Abort job |
Migration Checklist
- [ ] Extract job structure, list steps in order, identify programs/procedures, document COND/IF logic
- [ ] Map input/output datasets, identify temp datasets, document GDG usage, track data dependencies
- [ ] Convert COND to normal logic (INVERT!), translate IF/THEN/ELSE, handle error paths
- [ ] Choose target (Spring Batch/Airflow/shell), define job structure, implement steps, add monitoring
- [ ] Test normal path, error conditions, conditional branches with production-like data
- [ ] Document job purpose, schedule, dependencies, special requirements
Critical Tips
1. COND is INVERTED - step runs when condition is FALSE! Draw truth tables if needed. 2. Return codes: 0=success, 4=warning (OK), 8+=error 3. Data dependencies: Carefully map to avoid race conditions 4. Restart capability: Implement checkpointing if needed 5. Monitoring: Add logging and alerting to modern workflows
Output Structure
Provide: Job overview, step sequence, data flow, conditional logic, migration target, workflow definition, migration estimate, action items.
Advanced Topics
For detailed conversion rules and patterns, see:
- [pseudocode-jcl-rules.md](references/pseudocode-jcl-rules.md) - Comprehensive JCL to pseudocode conversion rules including element mapping, return codes, DISP parameters, translation patterns, and COND logic handling
- [pseudocode-common-rules.md](references/pseudocode-common-rules.md) - Common pseudocode syntax and conventions applicable to all languages
- [testing-strategy.md](references/testing-strategy.md) - Comprehensive testing approach including unit tests, integration tests, parallel validation, and data-driven testing for migrated workflows
- [transaction-handling.md](references/transaction-handling.md) - Transaction management, rollback strategies, and ACID compliance for batch jobs
- [messaging-integration.md](references/messaging-integration.md) - Message queue integration patterns (MQ, JMS, Kafka) for event-driven workflows
- [performance-patterns.md](references/performance-patterns.md) - Batch processing optimization, memory management, parallel processing, and performance tuning
Tools and Scripts
All scripts support cross-platform execution (Windows PowerShell, bash):
analyze-dependencies.sh/ps1- Generate dependency graph in JSON format showing job-to-job, job-to-dataset, and procedure dependenciesextract-structure.py- Parse JCL files and extract structure (job cards, steps, DD statements, COND logic) to JSONgenerate-java-classes.py- Generate Java POJOs from data structures for Spring Batch item readers/writersestimate-complexity.py- Calculate migration complexity score based on steps, conditional logic, procedures, and data dependencies
Scripts use standard libraries only and output JSON for easy integration with CI/CD pipelines and migration tracking tools.
Integration
Works with job schedulers (Control-M, cron), workflow platforms (Spring Batch, Airflow, K8s), monitoring tools, version control, and CI/CD pipelines.
package {{PACKAGE_NAME}};
import java.math.BigDecimal;
import java.time.LocalDate;
import java.util.List;
import java.util.ArrayList;
/**
* Generated from JCL job: {{JOB_NAME}}
*
* Migration Date: {{GENERATION_DATE}}
* Original Source: {{SOURCE_FILE}}
*
* This class represents a Spring Batch job configuration equivalent to the JCL job.
* Auto-generated - review and adjust as needed for your application.
*/
public class {{CLASS_NAME}} {
// ========================================
// Configuration Properties (from JCL parameters)
// ========================================
{{FIELD_DECLARATIONS}}
// ========================================
// Constructors
// ========================================
public {{CLASS_NAME}}() {
// Default constructor
}
public {{CLASS_NAME}}({{CONSTRUCTOR_PARAMETERS}}) {
{{CONSTRUCTOR_ASSIGNMENTS}}
}
// ========================================
// Getters and Setters
// ========================================
{{GETTERS_AND_SETTERS}}
// ========================================
// Job Steps (from JCL EXEC statements)
// ========================================
{{BUSINESS_METHODS}}
// ========================================
// Validation Methods
// ========================================
/**
* Validates the job configuration.
* Implements JCL parameter validation rules.
*
* @return true if valid, false otherwise
*/
public boolean isValid() {
// TODO: Implement validation logic from COBOL
return true;
}
// ========================================
// Utility Methods
// ========================================
@Override
public String toString() {
return "{{CLASS_NAME}}{" +
{{TO_STRING_FIELDS}} +
"}";
}
@Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
{{CLASS_NAME}} that = ({{CLASS_NAME}}) o;
// TODO: Implement equality check
return false;
}
@Override
public int hashCode() {
// TODO: Implement hash code
return 0;
}
}
JCL Job Migration Report
Program: {{PROGRAM_NAME}} Migration Date: {{MIGRATION_DATE}} Analyst: {{ANALYST_NAME}} Status: {{STATUS}}
---
1. Executive Summary
Program Overview
- Purpose: {{PROGRAM_PURPOSE}}
- Type: {{PROGRAM_TYPE}} (Batch/Online/Utility)
- Lines of Code: {{LOC}}
- Complexity: {{COMPLEXITY_LEVEL}}
- Estimated Effort: {{EFFORT_DAYS}} person-days
Migration Recommendation
{{RECOMMENDATION}}
---
2. Source Analysis
Program Structure
Divisions
- IDENTIFICATION DIVISION: Line {{ID_DIV_LINE}}
- ENVIRONMENT DIVISION: Line {{ENV_DIV_LINE}}
- DATA DIVISION: Line {{DATA_DIV_LINE}}
- PROCEDURE DIVISION: Line {{PROC_DIV_LINE}}
Key Data Structures
| Structure | Type | Lines | Description |
|---|
{{DATA_STRUCTURES_TABLE}}
Copybooks Used
| Copybook | Purpose | Lines |
|---|
{{COPYBOOKS_TABLE}}
Business Logic Summary
{{BUSINESS_LOGIC_SUMMARY}}
Control Flow
{{CONTROL_FLOW_DESCRIPTION}}
---
3. Dependencies
Program Calls
| Called Program | Purpose | Frequency |
|---|
{{PROGRAM_CALLS_TABLE}}
File Operations
| File Name | Access Mode | Operations |
|---|
{{FILE_OPERATIONS_TABLE}}
Database Operations
| Table | Operations | Estimated Rows |
|---|
{{DATABASE_OPERATIONS_TABLE}}
Dependency Graph
{{DEPENDENCY_GRAPH}}---
4. Java Design
Proposed Architecture
Package Structure
{{PACKAGE_STRUCTURE}}Class Design
| Java Class | Purpose | COBOL Equivalent |
|---|
{{CLASS_DESIGN_TABLE}}
Method Signatures
{{METHOD_SIGNATURES}}Data Model
{{DATA_MODEL_DESCRIPTION}}
Service Layer
{{SERVICE_LAYER_DESCRIPTION}}
---
5. Complexity Assessment
Metrics
- Cyclomatic Complexity: {{CYCLOMATIC_COMPLEXITY}}
- Number of Paragraphs: {{PARAGRAPH_COUNT}}
- External Dependencies: {{DEPENDENCY_COUNT}}
- File Operations: {{FILE_OP_COUNT}}
- Database Operations: {{DB_OP_COUNT}}
- Complexity Score: {{COMPLEXITY_SCORE}}
Risk Factors
{{RISK_FACTORS_LIST}}
Technical Challenges
1. {{CHALLENGE_1}} 2. {{CHALLENGE_2}} 3. {{CHALLENGE_3}}
---
6. Migration Strategy
Approach
{{MIGRATION_APPROACH}}
Phase 1: Preparation ({{PHASE1_DAYS}} days)
- [ ] {{PREP_TASK_1}}
- [ ] {{PREP_TASK_2}}
- [ ] {{PREP_TASK_3}}
Phase 2: Implementation ({{PHASE2_DAYS}} days)
- [ ] {{IMPL_TASK_1}}
- [ ] {{IMPL_TASK_2}}
- [ ] {{IMPL_TASK_3}}
Phase 3: Testing ({{PHASE3_DAYS}} days)
- [ ] {{TEST_TASK_1}}
- [ ] {{TEST_TASK_2}}
- [ ] {{TEST_TASK_3}}
Phase 4: Deployment ({{PHASE4_DAYS}} days)
- [ ] {{DEPLOY_TASK_1}}
- [ ] {{DEPLOY_TASK_2}}
- [ ] {{DEPLOY_TASK_3}}
---
7. Testing Plan
Unit Testing
{{UNIT_TEST_PLAN}}
Integration Testing
{{INTEGRATION_TEST_PLAN}}
Parallel Run Testing
{{PARALLEL_RUN_PLAN}}
Performance Testing
{{PERFORMANCE_TEST_PLAN}}
---
8. Data Migration
Data Structures to Migrate
{{DATA_MIGRATION_STRUCTURES}}
Migration Scripts Required
- [ ] {{MIGRATION_SCRIPT_1}}
- [ ] {{MIGRATION_SCRIPT_2}}
- [ ] {{MIGRATION_SCRIPT_3}}
Validation Approach
{{VALIDATION_APPROACH}}
---
9. Implementation Notes
Special Considerations
{{SPECIAL_CONSIDERATIONS}}
Code Patterns
{{CODE_PATTERNS}}
Performance Optimization
{{PERFORMANCE_NOTES}}
---
10. Timeline and Resources
Schedule
| Phase | Start Date | End Date | Duration |
|---|---|---|---|
| Preparation | {{P1_START}} | {{P1_END}} | {{P1_DURATION}} |
| Implementation | {{P2_START}} | {{P2_END}} | {{P2_DURATION}} |
| Testing | {{P3_START}} | {{P3_END}} | {{P3_DURATION}} |
| Deployment | {{P4_START}} | {{P4_END}} | {{P4_DURATION}} |
Resource Requirements
- Java Developers: {{JAVA_DEV_COUNT}}
- COBOL Analysts: {{COBOL_ANALYST_COUNT}}
- QA Engineers: {{QA_COUNT}}
- DevOps: {{DEVOPS_COUNT}}
Dependencies
- [ ] {{EXTERNAL_DEPENDENCY_1}}
- [ ] {{EXTERNAL_DEPENDENCY_2}}
- [ ] {{EXTERNAL_DEPENDENCY_3}}
---
11. Success Criteria
Functional
- [ ] All business logic correctly migrated
- [ ] 100% test coverage for critical paths
- [ ] Parallel run matches COBOL output (99.9%+)
Non-Functional
- [ ] Performance meets or exceeds COBOL
- [ ] Response time < {{RESPONSE_TIME_TARGET}}ms
- [ ] Throughput >= {{THROUGHPUT_TARGET}} TPS
Quality
- [ ] Code review completed
- [ ] Documentation complete
- [ ] Knowledge transfer completed
---
12. Appendices
A. COBOL Source Reference
{{COBOL_SOURCE_EXCERPTS}}
B. Generated Java Code Samples
{{JAVA_CODE_SAMPLES}}
C. Test Cases
{{TEST_CASES}}
D. References
- COBOL Program:
{{COBOL_FILE_PATH}} - Design Document:
{{DESIGN_DOC_PATH}} - Test Data:
{{TEST_DATA_PATH}}
---
Report Generated: {{REPORT_DATE}} Tool: COBOL Migration Analyzer Agent Skill v1.0.0
Messaging Integration Patterns
Guide for migrating mainframe messaging patterns (MQ, CICS transient data) to modern Java messaging.
Overview
Mainframe systems use IBM MQ, CICS queues, and transient data for asynchronous communication. This guide shows how to migrate these patterns to modern Java messaging frameworks.
Common Mainframe Patterns
1. IBM MQ Messages
COBOL Example:
EXEC CICS WRITEQ TS
QUEUE('ORDERQ')
FROM(ORDER-MESSAGE)
LENGTH(ORDER-LENGTH)
END-EXEC.
EXEC CICS READQ TS
QUEUE('ORDERQ')
INTO(ORDER-MESSAGE)
LENGTH(ORDER-LENGTH)
END-EXEC.2. CICS Transient Data
EXEC CICS WRITEQ TD
QUEUE('LOGG')
FROM(LOG-MESSAGE)
LENGTH(LOG-LENGTH)
END-EXEC.Java Messaging Patterns
Pattern 1: Spring JMS with ActiveMQ
Configuration:
@Configuration
@EnableJms
public class JmsConfig {
@Bean
public ConnectionFactory connectionFactory() {
return new ActiveMQConnectionFactory("tcp://localhost:61616");
}
@Bean
public JmsTemplate jmsTemplate(ConnectionFactory connectionFactory) {
JmsTemplate template = new JmsTemplate(connectionFactory);
template.setDefaultDestinationName("order.queue");
return template;
}
}Sender (equivalent to WRITEQ):
@Service
public class OrderMessageSender {
@Autowired
private JmsTemplate jmsTemplate;
public void sendOrder(Order order) {
jmsTemplate.convertAndSend("order.queue", order, message -> {
message.setStringProperty("orderType", order.getType());
message.setStringProperty("priority", order.getPriority());
return message;
});
}
}Receiver (equivalent to READQ):
@Service
public class OrderMessageReceiver {
@JmsListener(destination = "order.queue")
public void receiveOrder(Order order) {
log.info("Received order: {}", order.getId());
processOrder(order);
}
// Manual receive (blocking)
public Order receiveOrderManually() {
return (Order) jmsTemplate.receiveAndConvert("order.queue");
}
}Pattern 2: Spring AMQP with RabbitMQ
Configuration:
@Configuration
public class RabbitConfig {
@Bean
public Queue orderQueue() {
return new Queue("order.queue", true); // durable
}
@Bean
public Exchange orderExchange() {
return new TopicExchange("order.exchange");
}
@Bean
public Binding binding(Queue queue, Exchange exchange) {
return BindingBuilder
.bind(queue)
.to(exchange)
.with("order.#")
.noargs();
}
@Bean
public RabbitTemplate rabbitTemplate(ConnectionFactory connectionFactory) {
RabbitTemplate template = new RabbitTemplate(connectionFactory);
template.setMessageConverter(new Jackson2JsonMessageConverter());
return template;
}
}Sender:
@Service
public class RabbitOrderSender {
@Autowired
private RabbitTemplate rabbitTemplate;
public void sendOrder(Order order) {
rabbitTemplate.convertAndSend(
"order.exchange",
"order.new",
order
);
}
}Receiver:
@Service
public class RabbitOrderReceiver {
@RabbitListener(queues = "order.queue")
public void handleOrder(Order order) {
processOrder(order);
}
}Pattern 3: Spring Kafka
Configuration:
@Configuration
public class KafkaConfig {
@Bean
public ProducerFactory<String, Order> producerFactory() {
Map<String, Object> config = new HashMap<>();
config.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
config.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
config.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, JsonSerializer.class);
return new DefaultKafkaProducerFactory<>(config);
}
@Bean
public KafkaTemplate<String, Order> kafkaTemplate() {
return new KafkaTemplate<>(producerFactory());
}
@Bean
public ConsumerFactory<String, Order> consumerFactory() {
Map<String, Object> config = new HashMap<>();
config.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
config.put(ConsumerConfig.GROUP_ID_CONFIG, "order-processor");
config.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
config.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, JsonDeserializer.class);
config.put(JsonDeserializer.TRUSTED_PACKAGES, "*");
return new DefaultKafkaConsumerFactory<>(config);
}
}Producer:
@Service
public class KafkaOrderProducer {
@Autowired
private KafkaTemplate<String, Order> kafkaTemplate;
public void sendOrder(Order order) {
kafkaTemplate.send("order-topic", order.getId(), order)
.addCallback(
success -> log.info("Order sent: {}", order.getId()),
failure -> log.error("Failed to send order", failure)
);
}
}Consumer:
@Service
public class KafkaOrderConsumer {
@KafkaListener(topics = "order-topic", groupId = "order-processor")
public void consume(Order order) {
log.info("Consumed order: {}", order.getId());
processOrder(order);
}
}Migration Patterns
Pattern 1: Request-Reply
COBOL/MQ:
* Send request
MOVE 'TEMP-REPLY-Q' TO MQMD-REPLYTOQ
CALL 'MQPUT' USING ...
* Wait for reply
CALL 'MQGET' USING MQMD MQGMO REPLYQ-NAME ...Java/JMS:
@Service
public class RequestReplyService {
@Autowired
private JmsTemplate jmsTemplate;
public Response sendRequest(Request request) {
return (Response) jmsTemplate.convertSendAndReceive(
"request.queue",
request,
message -> {
message.setJMSReplyTo(
new ActiveMQQueue("reply.queue")
);
return message;
}
);
}
}Pattern 2: Dead Letter Queue
Configuration:
@Bean
public ReplyingKafkaTemplate<String, Order, OrderResult> replyingKafkaTemplate(
ProducerFactory<String, Order> pf,
KafkaMessageListenerContainer<String, OrderResult> container) {
return new ReplyingKafkaTemplate<>(pf, container);
}
@Bean
public DeadLetterPublishingRecoverer deadLetterPublishingRecoverer(
KafkaTemplate<String, Order> template) {
return new DeadLetterPublishingRecoverer(template);
}Usage:
@KafkaListener(topics = "order-topic")
public void processOrder(Order order) {
try {
validateAndProcess(order);
} catch (ValidationException e) {
// Will be sent to DLQ automatically
throw new RuntimeException("Invalid order", e);
}
}Pattern 3: Message Retry
Configuration:
@Configuration
public class RetryConfig {
@Bean
public ErrorHandler errorHandler(KafkaTemplate<String, Order> template) {
// Retry 3 times with exponential backoff
DefaultErrorHandler handler = new DefaultErrorHandler(
new DeadLetterPublishingRecoverer(template),
new FixedBackOff(1000L, 3L)
);
handler.addNotRetryableExceptions(ValidationException.class);
return handler;
}
}Message Format Conversion
COBOL Fixed-Length to JSON
COBOL Message:
01 ORDER-MESSAGE.
05 ORDER-ID PIC X(10).
05 CUSTOMER-ID PIC X(10).
05 AMOUNT PIC 9(7)V99 COMP-3.
05 ORDER-DATE PIC 9(8).Java DTO:
@Data
public class OrderMessage {
private String orderId;
private String customerId;
private BigDecimal amount;
private LocalDate orderDate;
// Converter from COBOL format
public static OrderMessage fromCobolBytes(byte[] bytes) {
// Parse fixed-length format
String orderId = new String(bytes, 0, 10).trim();
String customerId = new String(bytes, 10, 10).trim();
BigDecimal amount = CobolConverter.fromPackedDecimal(bytes, 20, 5);
LocalDate orderDate = CobolConverter.fromCobolDate(bytes, 25, 8);
return new OrderMessage(orderId, customerId, amount, orderDate);
}
}Transaction Management
XA Transactions with Messaging
@Configuration
@EnableTransactionManagement
public class XAMessagingConfig {
@Bean
public JmsTransactionManager transactionManager(
ConnectionFactory connectionFactory) {
return new JmsTransactionManager(connectionFactory);
}
}
@Service
public class TransactionalMessagingService {
@Transactional
public void processWithTransaction(Order order) {
// Save to database
orderRepository.save(order);
// Send message (part of same transaction)
jmsTemplate.convertAndSend("order.processed", order);
// Both commit or rollback together
}
}Monitoring and Error Handling
Message Metrics
@Component
public class MessagingMetrics {
private final Counter messagesReceived;
private final Counter messagesFailed;
private final Timer processingTime;
public MessagingMetrics(MeterRegistry registry) {
this.messagesReceived = registry.counter("messages.received");
this.messagesFailed = registry.counter("messages.failed");
this.processingTime = registry.timer("messages.processing.time");
}
@Around("@annotation(JmsListener)")
public Object monitorMessageProcessing(ProceedingJoinPoint pjp) throws Throwable {
messagesReceived.increment();
Timer.Sample sample = Timer.start();
try {
Object result = pjp.proceed();
sample.stop(processingTime);
return result;
} catch (Exception e) {
messagesFailed.increment();
throw e;
}
}
}Error Handling
@Service
public class MessagingErrorHandler {
@JmsListener(destination = "order.queue")
public void handleOrder(Order order, @Header("JMSRedelivered") boolean redelivered) {
try {
processOrder(order);
} catch (Exception e) {
if (redelivered) {
// Already retried, send to DLQ
sendToDeadLetterQueue(order, e);
} else {
// First failure, throw to trigger redelivery
throw new RuntimeException("Processing failed", e);
}
}
}
}Migration Checklist
- [ ] Identify all MQ queue usages
- [ ] Map CICS transient data queues
- [ ] Choose appropriate messaging technology (JMS/AMQP/Kafka)
- [ ] Design message formats (JSON vs. binary)
- [ ] Implement message converters
- [ ] Configure error handling and DLQ
- [ ] Set up monitoring and alerting
- [ ] Test message ordering guarantees
- [ ] Verify transaction behavior
- [ ] Performance test under load
- [ ] Document migration patterns
Performance Patterns for Migrated Code
This document provides guidance on optimizing Java code migrated from COBOL, addressing common performance challenges.
Overview
COBOL programs often process large volumes of data efficiently using mainframe-optimized patterns. When migrating to Java, it's important to maintain or improve performance while adapting to modern architectures.
Common Performance Patterns
1. Batch Processing with Streams
COBOL Pattern: Sequential file processing Java Solution: Use Java Streams for efficient batch processing
// Instead of loading all records into memory
List<Record> records = loadAllRecords(); // DON'T
// Use streaming
try (Stream<String> lines = Files.lines(path)) {
lines.map(this::parseRecord)
.filter(this::isValid)
.forEach(this::processRecord);
}2. Database Batch Operations
COBOL Pattern: Cursor processing with commits every N records Java Solution: JDBC batch updates
try (PreparedStatement pstmt = conn.prepareStatement(sql)) {
for (Record record : records) {
pstmt.setString(1, record.getId());
pstmt.setString(2, record.getName());
pstmt.addBatch();
if (++count % 1000 == 0) {
pstmt.executeBatch();
conn.commit();
}
}
pstmt.executeBatch();
conn.commit();
}3. Memory Management
Challenge: COBOL's fixed memory model vs. Java's heap Solution: Process in chunks, use pagination
public void processLargeFile(Path file) {
int batchSize = 1000;
List<Record> batch = new ArrayList<>(batchSize);
try (Stream<String> lines = Files.lines(file)) {
lines.forEach(line -> {
batch.add(parseRecord(line));
if (batch.size() >= batchSize) {
processBatch(new ArrayList<>(batch));
batch.clear();
}
});
if (!batch.isEmpty()) {
processBatch(batch);
}
}
}4. Parallel Processing
COBOL Pattern: Single-threaded sequential processing Java Solution: Parallel streams for CPU-bound operations
// For independent record processing
records.parallelStream()
.map(this::transform)
.forEach(this::save);
// Control parallelism
ForkJoinPool customPool = new ForkJoinPool(4);
customPool.submit(() ->
records.parallelStream()
.forEach(this::process)
).get();5. String Operations
COBOL Pattern: Fixed-length strings with spaces Java Solution: Efficient string handling
// Avoid creating many temporary strings
StringBuilder sb = new StringBuilder();
for (Record r : records) {
sb.append(r.getId()).append('|')
.append(r.getName()).append('\n');
}
String output = sb.toString();
// For fixed-length COBOL fields
String padded = String.format("%-20s", value); // Left-padded
String numeric = String.format("%010d", number); // Zero-padded6. Caching Lookup Tables
COBOL Pattern: In-memory tables loaded at startup Java Solution: Use efficient caching
// Simple cache
private final Map<String, RateEntry> rateCache = new HashMap<>();
// Caffeine cache with eviction
LoadingCache<String, RateEntry> cache = Caffeine.newBuilder()
.maximumSize(10_000)
.expireAfterWrite(1, TimeUnit.HOURS)
.build(key -> loadRate(key));7. File I/O Optimization
COBOL Pattern: Blocked records for efficient I/O Java Solution: Buffered I/O with appropriate buffer sizes
// Reading
try (BufferedReader reader = new BufferedReader(
new FileReader(file), 8192 * 4)) { // 32KB buffer
String line;
while ((line = reader.readLine()) != null) {
processLine(line);
}
}
// Writing
try (BufferedWriter writer = new BufferedWriter(
new FileWriter(file), 8192 * 4)) {
for (Record r : records) {
writer.write(r.toLine());
writer.newLine();
}
}Performance Testing
Benchmark Template
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@State(Scope.Thread)
public class MigrationBenchmark {
@Param({"1000", "10000", "100000"})
private int recordCount;
@Setup
public void setup() {
// Initialize test data
}
@Benchmark
public void testProcessing() {
// Your processing logic
}
}Best Practices
1. Profile Before Optimizing: Use JProfiler, YourKit, or JFR 2. Set Realistic Goals: Match or exceed COBOL performance 3. Test with Production Volumes: Use representative data sizes 4. Monitor JVM Metrics: Heap, GC, thread pools 5. Use Appropriate Data Structures: ArrayList vs. LinkedList, HashMap vs. TreeMap 6. Minimize Object Creation: Reuse objects in tight loops 7. Consider Parallel Processing: But measure - not always faster 8. Optimize Database Access: Use connection pooling, prepared statements
Common Pitfalls
❌ DON'T: Load entire files into memory ✅ DO: Stream and process incrementally
❌ DON'T: Use + for string concatenation in loops ✅ DO: Use StringBuilder or StringJoiner
❌ DON'T: Create new SimpleDateFormat in loops (not thread-safe) ✅ DO: Use DateTimeFormatter (thread-safe) or ThreadLocal
❌ DON'T: Ignore connection pooling ✅ DO: Use HikariCP or similar
❌ DON'T: Assume parallel is always faster ✅ DO: Benchmark and measure
Monitoring Migration Performance
// Add metrics
public class ProcessingService {
private final Timer processingTimer;
private final Counter recordCounter;
public void processRecords(List<Record> records) {
Timer.Sample sample = Timer.start();
try {
records.forEach(this::process);
recordCounter.increment(records.size());
} finally {
sample.stop(processingTimer);
}
}
}Pseudocode Common Rules - All Languages
Naming Conventions
- Variables: camelCase (
inputCount,customerName) - Types/Structures: PascalCase (
CustomerRecord,OrderDetail) - Constants: UPPER_SNAKE_CASE (
MAX_RECORDS,TAX_RATE) - Functions/Procedures: PascalCase with verb (
ProcessRecord,CalculateTotal)
Data Types
INTEGER- whole numbersDECIMAL(n,m)- financial precision (n digits, m decimals)STRING[n]- text (max n characters)BOOLEAN- true/falseDATE/DATETIME- dates/timesARRAY[n] OF TYPE- arraysSTRUCTURE- composite types
Pseudocode Syntax
Structure Definition
STRUCTURE StructureName:
field: TYPE[length] // Description
END STRUCTUREConstants
CONSTANTS:
NAME = value
END CONSTANTSFunctions/Procedures
FUNCTION Name(param: TYPE) RETURNS TYPE
BEGIN
RETURN value
END FUNCTION
PROCEDURE Name(param: TYPE)
BEGIN
statements
END PROCEDUREControl Flow
IF condition THEN ... ELSE ... END IF
WHILE condition DO ... END WHILE
FOR i FROM start TO end BY step DO ... END FOR
SWITCH expr: CASE val: ... BREAK; DEFAULT: ... END SWITCHFile Operations
file = OPEN(path) FOR READING|WRITING|APPENDING
record = READ_RECORD(file)
WRITE_RECORD(file, record)
CLOSE(file)
IF END_OF_FILE(file) THEN ...Error Handling
TRY:
statements
CATCH ExceptionType:
handler
FINALLY:
cleanup
END TRYFinancial Precision Rule
CRITICAL: Always use DECIMAL(n,m) for money. Never use floating point.
amount: DECIMAL(15,2)
result = ROUND(calculation, 2) // Use HALF_UP roundingMermaid Flowchart Template
````
flowchart TD
Start([Start])
Process[Process Step]
Decision{Condition?}
End([End])
Start --> Process
Process --> Decision
Decision -->|Yes| End
Decision -->|No| Process````
Document Structure Template
# [PROGRAM-NAME] - Description
## Program Overview
- Purpose, Input, Output, Original Source
## Data Structures
[Structures and constants]
## Main Algorithm
[High-level flow]
## Core Processing Logic
[Detailed procedures]
## Flowchart (Mermaid)
[Diagram]
## Decision Logic / Special Cases / Error Handling
[Business rules and edge cases]
## Example Traces / Testing / Integration Points
[Walkthroughs and dependencies]JCL Translation Rules
Prerequisites: Read PSEUDOCODE-COMMON-RULES.md for syntax, naming, and structure.
---
Element Mapping
| JCL Element | Pseudocode Equivalent |
|---|---|
//JOB | Job Definition |
//EXEC PGM= | Execute Program |
//EXEC PROC= | Call Procedure |
//DD | Dataset Definition |
//COND | Conditional Execution |
//IF/THEN/ELSE | IF-THEN-ELSE logic |
//PROC | Procedure Definition |
Return Code Mapping
| RC | Meaning | Pseudocode |
|---|---|---|
| 0 | Success | Success |
| 4 | Warning | Warning (continue) |
| 8 | Error | Error (may stop) |
| 12 | Severe error | Severe Error (stop) |
| 16 | Fatal error | Fatal Error (abort) |
DISP Parameter Translation
| DISP | Status | Normal End | Abnormal End | Pseudocode |
|---|---|---|---|---|
SHR | Shared | Keep | Keep | READ_SHARED(dataset) |
OLD | Exclusive | Keep | Keep | READ_EXCLUSIVE(dataset) |
NEW | Create | CATLG | DELETE | CREATE(dataset) |
MOD | Append | Keep | Keep | APPEND(dataset) |
Translation Patterns
Job Step Sequence
//STEP010 EXEC PGM=PROG1
//STEP020 EXEC PGM=PROG2
//STEP030 EXEC PGM=PROG3→
JOB JobName
BEGIN
STEP Step010
EXECUTE PROG1
IF RETURN_CODE > 0 THEN STOP
END STEP
STEP Step020
EXECUTE PROG2
IF RETURN_CODE > 0 THEN STOP
END STEP
STEP Step030
EXECUTE PROG3
END STEP
END JOBCOND Logic (⚠️ INVERTED!)
//STEP020 EXEC PGM=PROG2,COND=(0,NE)→
STEP Step020
IF PREVIOUS_RC = 0 THEN SKIP // INVERTED: run if condition FALSE!
EXECUTE PROG2
END STEPCRITICAL: COND logic is inverted - step runs if condition is FALSE!
IF/THEN/ELSE
//IF1 IF RC = 0 THEN
//STEP020 EXEC PGM=PROG2
//ENDIF→
IF PREVIOUS_RC = 0 THEN
STEP Step020
EXECUTE PROG2
END STEP
END IFDD Statement
//INPUT DD DSN=PROD.DATA.FILE,DISP=SHR
//OUTPUT DD DSN=PROD.OUTPUT.FILE,DISP=(NEW,CATLG,DELETE)→
input = OPEN_DATASET("PROD.DATA.FILE", SHARED, READ)
output = CREATE_DATASET("PROD.OUTPUT.FILE")
// Normal end: CATALOG
// Abnormal end: DELETEProc Call with Symbolics
//PROC1 PROC MEMBER=,INFILE=
//STEP1 EXEC PGM=PROG1
//SYSIN DD DSN=&MEMBER,DISP=SHR
//INPUT DD DSN=&INFILE,DISP=SHR
// PEND
//CALLPROC EXEC PROC1,MEMBER=TEST.DATA,INFILE=PROD.FILE→
PROCEDURE Proc1(member: STRING, inFile: STRING)
BEGIN
STEP Step1
EXECUTE PROG1
sysin = OPEN_DATASET(member, SHARED)
input = OPEN_DATASET(inFile, SHARED)
END STEP
END PROCEDURE
CALL Proc1("TEST.DATA", "PROD.FILE")Critical Rules
1. COND is inverted: Step runs when condition is FALSE (opposite of normal IF) 2. Return codes: 0=success, 4=warning (OK), 8+=error 3. DISP triple: (status, normal-end, abnormal-end) 4. Sequential execution: Steps run in order unless COND/IF skips 5. Dataset lifecycle: Track NEW→CATLG→DELETE transitions 6. Symbolic parameters: &VAR replaced with actual values
Translation Workflow
1. Parse JOB card → Job Definition 2. For each STEP: Extract PGM/PROC → Execution steps 3. Convert COND → Inverted IF logic (!condition) 4. Map IF/THEN/ELSE → Standard conditionals 5. Translate DD → Dataset operations 6. Convert PROC → Procedure definitions 7. Generate Mermaid flowchart showing step dependencies 8. Document return code handling
Reference: IBM z/OS MVS JCL Reference, JCL User's Guide
Testing Strategy for COBOL to Java Migration
Comprehensive testing approach to ensure functional equivalence and quality during migration.
Testing Pyramid for Migration
/\
/ \
/ E2E\
/------\
/ Integ \
/----------\
/ Unit \
/--------------\1. Unit Testing
Testing Converted Business Logic
Goal: Verify each Java method matches COBOL paragraph behavior
@Test
void testCalculateBonus_seniorEmployee() {
// Given
Employee emp = new Employee();
emp.setYearsOfService(12);
emp.setSalary(new BigDecimal("50000"));
// When
BigDecimal bonus = bonusCalculator.calculate(emp);
// Then
assertEquals(new BigDecimal("7500.00"), bonus);
}
@Test
void testCalculateBonus_midLevelEmployee() {
Employee emp = new Employee();
emp.setYearsOfService(7);
emp.setSalary(new BigDecimal("40000"));
BigDecimal bonus = bonusCalculator.calculate(emp);
assertEquals(new BigDecimal("4000.00"), bonus);
}
@Test
void testCalculateBonus_juniorEmployee() {
Employee emp = new Employee();
emp.setYearsOfService(2);
emp.setSalary(new BigDecimal("30000"));
BigDecimal bonus = bonusCalculator.calculate(emp);
assertEquals(new BigDecimal("1500.00"), bonus);
}Testing Data Transformations
@Test
void testCobolToJavaDataConversion() {
// COBOL PIC S9(7)V99 COMP-3
String cobolPackedDecimal = "0000012345C"; // 123.45
BigDecimal result = converter.fromPackedDecimal(cobolPackedDecimal);
assertEquals(new BigDecimal("123.45"), result);
}
@Test
void testDateConversion() {
// COBOL date format: YYYYMMDD
String cobolDate = "20260113";
LocalDate result = converter.fromCobolDate(cobolDate);
assertEquals(LocalDate.of(2026, 1, 13), result);
}Parameterized Tests for Edge Cases
@ParameterizedTest
@CsvSource({
"0, 0.00",
"1, 1500.00",
"5, 2000.00",
"6, 4000.00",
"10, 4000.00",
"11, 7500.00",
"20, 7500.00"
})
void testBonusCalculation_allRanges(int years, String expectedBonus) {
Employee emp = new Employee();
emp.setYearsOfService(years);
emp.setSalary(new BigDecimal("50000"));
BigDecimal bonus = bonusCalculator.calculate(emp);
assertEquals(new BigDecimal(expectedBonus), bonus);
}2. Integration Testing
Database Operations
@SpringBootTest
@Transactional
class CustomerRepositoryTest {
@Autowired
private CustomerRepository customerRepo;
@Test
void testSaveAndRetrieve() {
// Given
Customer customer = new Customer();
customer.setId("CUST001");
customer.setName("Test Customer");
// When
customerRepo.save(customer);
entityManager.flush();
entityManager.clear();
// Then
Optional<Customer> retrieved = customerRepo.findById("CUST001");
assertTrue(retrieved.isPresent());
assertEquals("Test Customer", retrieved.get().getName());
}
}File Processing
@SpringBootTest
class FileProcessorTest {
@Autowired
private FileProcessor processor;
@TempDir
Path tempDir;
@Test
void testProcessInputFile() throws IOException {
// Given
Path inputFile = tempDir.resolve("input.txt");
Files.write(inputFile, Arrays.asList(
"CUST001John Doe 00012345",
"CUST002Jane Smith 00067890"
));
// When
ProcessingResult result = processor.process(inputFile);
// Then
assertEquals(2, result.getProcessedCount());
assertEquals(0, result.getErrorCount());
}
}3. Parallel Run Testing
Strategy: Run Both Systems in Parallel
Goal: Prove functional equivalence with production data
@Service
public class ParallelRunService {
@Autowired
private LegacySystemAdapter legacyAdapter;
@Autowired
private NewJavaService newService;
@Autowired
private ResultComparator comparator;
public ProcessingResult processWithComparison(InputData input) {
// Run new system
Result newResult = newService.process(input);
// Run legacy system (asynchronously)
CompletableFuture<Result> legacyFuture =
CompletableFuture.supplyAsync(() -> legacyAdapter.process(input));
// Compare results
legacyFuture.thenAccept(legacyResult -> {
ComparisonResult comparison = comparator.compare(newResult, legacyResult);
if (!comparison.isMatch()) {
logDiscrepancy(input, newResult, legacyResult, comparison);
}
});
// Return new result (legacy is shadow)
return newResult;
}
}Result Comparison
@Service
public class ResultComparator {
public ComparisonResult compare(Result actual, Result expected) {
ComparisonResult result = new ComparisonResult();
// Compare numeric fields with tolerance
if (!compareDecimal(actual.getAmount(), expected.getAmount(), 0.01)) {
result.addDifference("amount", expected.getAmount(), actual.getAmount());
}
// Compare strings (trim for COBOL spaces)
if (!actual.getName().trim().equals(expected.getName().trim())) {
result.addDifference("name", expected.getName(), actual.getName());
}
// Compare dates
if (!actual.getDate().equals(expected.getDate())) {
result.addDifference("date", expected.getDate(), actual.getDate());
}
return result;
}
private boolean compareDecimal(BigDecimal a, BigDecimal b, double tolerance) {
return a.subtract(b).abs().doubleValue() <= tolerance;
}
}4. End-to-End Testing
Full Workflow Testing
@SpringBootTest
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
class OrderWorkflowE2ETest {
@Test
@Order(1)
void test_completeOrderWorkflow() {
// Step 1: Create order
OrderRequest request = new OrderRequest();
request.setCustomerId("CUST001");
request.addItem("ITEM001", 2);
Order order = orderService.createOrder(request);
assertNotNull(order.getId());
// Step 2: Process payment
PaymentResult payment = paymentService.process(order);
assertEquals(PaymentStatus.APPROVED, payment.getStatus());
// Step 3: Ship order
ShipmentResult shipment = shippingService.ship(order);
assertEquals(ShipmentStatus.SHIPPED, shipment.getStatus());
// Step 4: Verify final state
Order finalOrder = orderService.getOrder(order.getId());
assertEquals(OrderStatus.COMPLETED, finalOrder.getStatus());
}
}5. Performance Testing
Baseline Comparison
@State(Scope.Thread)
@BenchmarkMode(Mode.Throughput)
@OutputTimeUnit(TimeUnit.SECONDS)
public class PerformanceBenchmark {
private List<Record> testData;
@Setup
public void setup() {
testData = generateTestData(10000);
}
@Benchmark
public void testJavaImplementation() {
javaService.process(testData);
}
// Compare with baseline from COBOL system
// Target: >= 90% of COBOL throughput
}Load Testing
@SpringBootTest
class LoadTest {
@Test
void testConcurrentProcessing() throws InterruptedException {
int threadCount = 10;
int iterationsPerThread = 100;
ExecutorService executor = Executors.newFixedThreadPool(threadCount);
List<Future<?>> futures = new ArrayList<>();
for (int i = 0; i < threadCount; i++) {
futures.add(executor.submit(() -> {
for (int j = 0; j < iterationsPerThread; j++) {
orderService.processOrder(createTestOrder());
}
}));
}
// Wait for completion
for (Future<?> future : futures) {
future.get();
}
executor.shutdown();
// Verify results
long totalOrders = orderRepository.count();
assertEquals(threadCount * iterationsPerThread, totalOrders);
}
}6. Data Migration Testing
Data Validation
@SpringBootTest
class DataMigrationTest {
@Test
void validateMigratedCustomers() {
// Load from legacy system
List<LegacyCustomer> legacyCustomers = legacyDataSource.getAllCustomers();
// Compare with migrated data
for (LegacyCustomer legacy : legacyCustomers) {
Customer migrated = customerRepo.findById(legacy.getId())
.orElseThrow(() -> new AssertionError("Customer not migrated: " + legacy.getId()));
assertEquals(legacy.getName().trim(), migrated.getName());
assertEquals(legacy.getBalance(), migrated.getBalance());
// ... more assertions
}
}
@Test
void verifyDataIntegrity() {
// Check referential integrity
List<Order> orders = orderRepo.findAll();
for (Order order : orders) {
assertTrue(customerRepo.existsById(order.getCustomerId()),
"Order references non-existent customer: " + order.getId());
}
}
}7. Regression Testing
Automated Regression Suite
@SpringBootTest
@Tag("regression")
class RegressionTestSuite {
@Nested
@DisplayName("Customer Management")
class CustomerTests {
@Test
void testCreateCustomer() { /* ... */ }
@Test
void testUpdateCustomer() { /* ... */ }
@Test
void testDeleteCustomer() { /* ... */ }
}
@Nested
@DisplayName("Order Processing")
class OrderTests {
// ... order tests
}
}Test Data Strategy
Using Production-Like Data
@TestConfiguration
public class TestDataConfig {
@Bean
public TestDataGenerator testDataGenerator() {
return TestDataGenerator.builder()
.withCustomerCount(1000)
.withOrderCount(5000)
.withProductCount(100)
.withRealisticDistribution(true)
.build();
}
}Anonymizing Production Data
public class DataAnonymizer {
public Customer anonymize(Customer customer) {
Customer anonymized = new Customer();
anonymized.setId(customer.getId());
anonymized.setName(generateFakeName());
anonymized.setEmail(generateFakeEmail());
anonymized.setBalance(customer.getBalance()); // Keep financial data
return anonymized;
}
}Test Automation
CI/CD Integration
# .github/workflows/test.yml
name: Migration Tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up JDK
uses: actions/setup-java@v2
with:
java-version: '17'
- name: Run Unit Tests
run: ./mvnw test
- name: Run Integration Tests
run: ./mvnw verify -P integration-tests
- name: Run E2E Tests
run: ./mvnw verify -P e2e-tests
- name: Generate Coverage Report
run: ./mvnw jacoco:reportCoverage Goals
- Unit Test Coverage: ≥ 80% line coverage
- Integration Test Coverage: All APIs and database operations
- E2E Coverage: All critical business workflows
- Parallel Run Coverage: 100% of transactions for 30 days
Test Documentation
Test Case Template
/**
* Test: Calculate bonus for senior employee
*
* Legacy Reference: PROGRAM_NAME, calculation step
*
* Business Rule: Employees with >10 years service get 15% bonus
*
* Test Data:
* - Years of service: 12
* - Salary: $50,000
*
* Expected Result: $7,500 bonus
*/
@Test
void testCalculateBonus_seniorEmployee() {
// Test implementation
}Monitoring Test Results
@ExtendWith(TestResultLogger.class)
class MonitoredTest {
@Test
void testWithMonitoring() {
// Test execution metrics logged automatically
}
}
class TestResultLogger implements TestWatcher {
@Override
public void testSuccessful(ExtensionContext context) {
log.info("Test passed: {}", context.getDisplayName());
metricsService.recordTestSuccess(context);
}
@Override
public void testFailed(ExtensionContext context, Throwable cause) {
log.error("Test failed: {}", context.getDisplayName(), cause);
metricsService.recordTestFailure(context, cause);
}
}Checklist
- [ ] Unit tests for all business logic methods
- [ ] Integration tests for data access
- [ ] E2E tests for critical workflows
- [ ] Performance tests vs. baseline
- [ ] Parallel run for 30 days minimum
- [ ] Data migration validation
- [ ] Regression suite automated
- [ ] Test data anonymized
- [ ] CI/CD pipeline configured
- [ ] Coverage reports generated
- [ ] Test documentation complete
Transaction Handling in Migrated Applications
This document explains how to handle transactions when migrating from CICS/mainframe environments to Java.
Overview
COBOL programs running under CICS or IMS use implicit transaction management. In Java, transactions must be explicitly managed using appropriate frameworks and patterns.
CICS Transaction Concepts
CICS Transaction Model
Key Characteristics:
- Implicit transaction boundaries
- SYNCPOINT for explicit commits
- Automatic rollback on ABEND
- Resource coordination (files, databases, queues)
- Conversational vs. pseudo-conversational
Common CICS Commands
EXEC CICS SYNCPOINT END-EXEC.
EXEC CICS SYNCPOINT ROLLBACK END-EXEC.
EXEC CICS LINK PROGRAM(name) END-EXEC.
EXEC CICS RETURN END-EXEC.Java Transaction Patterns
1. Programmatic Transactions
For simple cases, use JDBC transactions:
Connection conn = dataSource.getConnection();
try {
conn.setAutoCommit(false);
// Perform operations
updateCustomer(conn, customer);
insertOrder(conn, order);
conn.commit();
} catch (Exception e) {
conn.rollback();
throw e;
} finally {
conn.close();
}2. Declarative Transactions (Spring)
For most migrations, use Spring's declarative approach:
@Service
@Transactional
public class OrderService {
@Autowired
private CustomerRepository customerRepo;
@Autowired
private OrderRepository orderRepo;
@Transactional
public void processOrder(OrderRequest request) {
// All operations in one transaction
Customer customer = customerRepo.findById(request.getCustomerId())
.orElseThrow(() -> new CustomerNotFoundException());
Order order = new Order();
order.setCustomer(customer);
order.setAmount(request.getAmount());
orderRepo.save(order);
// Transaction commits automatically on success
// Rolls back on unchecked exceptions
}
@Transactional(readOnly = true)
public Order getOrder(Long orderId) {
// Read-only optimization
return orderRepo.findById(orderId)
.orElseThrow(() -> new OrderNotFoundException());
}
}3. Transaction Propagation
Map CICS patterns to Spring propagation:
// CICS: New transaction (like EXEC CICS LINK SYNCONRETURN)
@Transactional(propagation = Propagation.REQUIRES_NEW)
public void processInNewTransaction() {
// Runs in separate transaction
}
// CICS: Join existing transaction
@Transactional(propagation = Propagation.REQUIRED)
public void processInCurrentTransaction() {
// Joins existing or creates new
}
// CICS: No transaction coordination
@Transactional(propagation = Propagation.NOT_SUPPORTED)
public void processWithoutTransaction() {
// Suspends any existing transaction
}4. Isolation Levels
Match CICS resource locking to SQL isolation:
// CICS UPDATE (exclusive lock)
@Transactional(isolation = Isolation.SERIALIZABLE)
public void updateWithExclusiveLock() {
// Prevents concurrent access
}
// CICS READ (shared lock)
@Transactional(isolation = Isolation.REPEATABLE_READ)
public void readWithConsistency() {
// Consistent reads within transaction
}Migration Patterns
Pattern 1: Batch SYNCPOINT
COBOL/CICS:
PERFORM VARYING WS-COUNTER FROM 1 BY 1
UNTIL WS-COUNTER > 1000
PERFORM PROCESS-RECORD
IF WS-COUNTER = 100
EXEC CICS SYNCPOINT END-EXEC
MOVE 0 TO WS-COUNTER
END-IF
END-PERFORM.Java:
@Service
public class BatchProcessor {
@Autowired
private TransactionTemplate transactionTemplate;
public void processBatch(List<Record> records) {
int batchSize = 100;
List<Record> batch = new ArrayList<>();
for (Record record : records) {
batch.add(record);
if (batch.size() >= batchSize) {
processBatchInTransaction(new ArrayList<>(batch));
batch.clear();
}
}
if (!batch.isEmpty()) {
processBatchInTransaction(batch);
}
}
private void processBatchInTransaction(List<Record> batch) {
transactionTemplate.execute(status -> {
batch.forEach(this::processRecord);
return null;
});
}
}Pattern 2: Compensating Transactions
Challenge: Multi-system transactions without XA Solution: Saga pattern
@Service
public class OrderSagaService {
public void processOrder(OrderRequest request) {
String sagaId = UUID.randomUUID().toString();
try {
// Step 1: Reserve inventory
String reservationId = inventoryService.reserve(request.getItems());
// Step 2: Charge payment
String paymentId = paymentService.charge(request.getPayment());
// Step 3: Create order
Order order = orderService.create(request, reservationId, paymentId);
// Success - commit all
inventoryService.confirm(reservationId);
paymentService.confirm(paymentId);
} catch (Exception e) {
// Compensate in reverse order
if (paymentId != null) {
paymentService.refund(paymentId);
}
if (reservationId != null) {
inventoryService.release(reservationId);
}
throw new OrderProcessingException("Order failed", e);
}
}
}Pattern 3: Distributed Transactions (XA)
For true 2-phase commit across resources:
@Configuration
public class XAConfig {
@Bean
public PlatformTransactionManager transactionManager() {
JtaTransactionManager tm = new JtaTransactionManager();
tm.setTransactionManager(atomikosTransactionManager());
return tm;
}
@Bean
public DataSource xaDataSource1() {
AtomikosDataSourceBean ds = new AtomikosDataSourceBean();
ds.setUniqueResourceName("db1");
ds.setXaDataSourceClassName("org.postgresql.xa.PGXADataSource");
// ... configuration
return ds;
}
@Bean
public DataSource xaDataSource2() {
AtomikosDataSourceBean ds = new AtomikosDataSourceBean();
ds.setUniqueResourceName("db2");
// ... configuration
return ds;
}
}
@Service
public class DistributedService {
@Transactional
public void updateBothDatabases() {
// Both updates in same XA transaction
db1Repository.update(data1);
db2Repository.update(data2);
// Atomikos coordinates 2PC
}
}Error Handling
CICS ABEND → Java Exception
COBOL/CICS:
IF ERROR-CONDITION
EXEC CICS ABEND ABCODE('APPL') END-EXEC
END-IF.Java:
@Transactional
public void process() {
if (errorCondition) {
// Unchecked exception triggers rollback
throw new ApplicationException("APPL");
}
}
// For checked exceptions that should rollback
@Transactional(rollbackFor = BusinessException.class)
public void processWithChecked() throws BusinessException {
if (errorCondition) {
throw new BusinessException("Error occurred");
}
}Testing Strategies
Unit Testing with Transactions
@SpringBootTest
@Transactional
class OrderServiceTest {
@Autowired
private OrderService orderService;
@Test
@Rollback // Default behavior
void testOrderProcessing() {
// Test runs in transaction
// Automatically rolled back after test
Order order = orderService.processOrder(request);
assertNotNull(order.getId());
}
@Test
@Commit // Explicitly commit for verification
void testOrderPersistence() {
Order order = orderService.processOrder(request);
// Verify in separate transaction
}
}Integration Testing
@SpringBootTest
@DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_EACH_TEST_METHOD)
class TransactionIntegrationTest {
@Test
void testRollbackOnError() {
assertThrows(BusinessException.class, () -> {
orderService.processInvalidOrder(invalidRequest);
});
// Verify rollback
assertFalse(orderRepository.findById(orderId).isPresent());
}
}Configuration Best Practices
Spring Boot Configuration
spring:
datasource:
hikari:
auto-commit: false
maximum-pool-size: 20
connection-timeout: 30000
jpa:
properties:
hibernate:
connection:
isolation: 2 # READ_COMMITTED
jdbc:
batch_size: 50
order_inserts: true
order_updates: trueTransaction Manager Tuning
@Configuration
public class TransactionConfig {
@Bean
public PlatformTransactionManager transactionManager(EntityManagerFactory emf) {
JpaTransactionManager tm = new JpaTransactionManager(emf);
tm.setDefaultTimeout(30); // 30 seconds
return tm;
}
}Monitoring and Debugging
Enable Transaction Logging
logging:
level:
org.springframework.transaction: DEBUG
org.springframework.orm.jpa: DEBUG
org.hibernate.engine.transaction: DEBUGTransaction Events
@Component
public class TransactionEventListener {
@TransactionalEventListener(phase = TransactionPhase.BEFORE_COMMIT)
public void beforeCommit(OrderCreatedEvent event) {
log.info("About to commit order: {}", event.getOrderId());
}
@TransactionalEventListener(phase = TransactionPhase.AFTER_COMPLETION)
public void afterCompletion(OrderCreatedEvent event) {
log.info("Transaction completed for order: {}", event.getOrderId());
}
}Common Issues and Solutions
Issue: Lost Updates
Problem: Concurrent modifications Solution: Use optimistic or pessimistic locking
@Entity
public class Account {
@Id
private Long id;
@Version // Optimistic locking
private Long version;
private BigDecimal balance;
}
// Or pessimistic locking
@Lock(LockModeType.PESSIMISTIC_WRITE)
Account findById(Long id);Issue: Long-Running Transactions
Problem: Locks held too long Solution: Minimize transaction scope
// Bad: Transaction spans user interaction
@Transactional
public void processOrderBad(OrderRequest request) {
Order order = createOrder(request);
sendEmail(order); // Slow!
updateInventory(order);
}
// Good: Transaction only for data operations
public void processOrderGood(OrderRequest request) {
Order order;
// Transaction scope
order = transactionTemplate.execute(status -> {
Order o = createOrder(request);
updateInventory(o);
return o;
});
// Outside transaction
sendEmail(order);
}Migration Checklist
- [ ] Identify all SYNCPOINT locations
- [ ] Map CICS transaction boundaries
- [ ] Determine isolation requirements
- [ ] Plan for distributed transactions
- [ ] Define compensating actions
- [ ] Implement error handling
- [ ] Add transaction monitoring
- [ ] Test rollback scenarios
- [ ] Performance test with load
- [ ] Document transaction flows
#!/usr/bin/env pwsh
#
# Analyze dependencies between JCL jobs, programs, and datasets.
# This script scans a directory of JCL jobs and generates a dependency graph.
# PowerShell version for Windows/Cross-platform support
param(
[string]$SourceDir = "."
)
$ErrorActionPreference = "Stop"
$ScriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path
$OutputFile = "dependencies.json"
Write-Host "Analyzing dependencies in: $SourceDir"
Write-Host "Output will be written to: $OutputFile"
# Initialize JSON structure
$initialJson = @{
programs = @()
dependencies = @()
copybooks = @()
files = @()
} | ConvertTo-Json -Depth 3
Set-Content -Path $OutputFile -Value $initialJson -Encoding UTF8
# Function to extract program name from COBOL file
function Get-ProgramName {
param([string]$FilePath)
$content = Get-Content $FilePath -ErrorAction SilentlyContinue
if ($content) {
foreach ($line in $content) {
if ($line -match 'PROGRAM-ID[\.\s]+([A-Za-z0-9-]+)') {
return $matches[1]
}
}
}
return (Get-Item $FilePath).BaseName
}
# Function to extract CALL statements
function Get-CallStatements {
param([string]$FilePath)
$calls = @()
$content = Get-Content $FilePath -ErrorAction SilentlyContinue
if ($content) {
foreach ($line in $content) {
if ($line -match 'CALL.*[''"]([A-Z0-9-]+)[''"]') {
$calls += $matches[1]
}
}
}
return $calls
}
# Function to extract COPY statements
function Get-CopyStatements {
param([string]$FilePath)
$copies = @()
$content = Get-Content $FilePath -ErrorAction SilentlyContinue
if ($content) {
foreach ($line in $content) {
if ($line -match 'COPY\s+([A-Z0-9-]+)') {
$copies += $matches[1] -replace '\.$', ''
}
}
}
return $copies
}
# Function to extract SELECT/ASSIGN file names
function Get-FileReferences {
param([string]$FilePath)
$fileRefs = @()
$content = Get-Content $FilePath -ErrorAction SilentlyContinue
if ($content) {
foreach ($line in $content) {
if ($line -match 'SELECT\s+([A-Z0-9-]+)') {
$fileRefs += $matches[1]
}
}
}
return $fileRefs
}
# Find all COBOL files
Write-Host "Scanning for COBOL files..."
$cobolFiles = Get-ChildItem -Path $SourceDir -Recurse -Include "*.cbl","*.CBL","*.cob","*.COB" -File -ErrorAction SilentlyContinue
if (-not $cobolFiles -or $cobolFiles.Count -eq 0) {
Write-Host "No COBOL files found in $SourceDir"
exit 1
}
Write-Host "Found $($cobolFiles.Count) COBOL files"
# Collections for data
$programs = @()
$dependencies = @()
$copybooks = @()
$files = @()
# Process each COBOL file
foreach ($cobolFile in $cobolFiles) {
Write-Host "Processing: $($cobolFile.FullName)"
$programName = Get-ProgramName -FilePath $cobolFile.FullName
# Add program to list
$programs += @{
name = $programName
file = $cobolFile.FullName
}
# Extract calls
$calls = Get-CallStatements -FilePath $cobolFile.FullName
foreach ($calledProgram in $calls) {
if ($calledProgram) {
$dependencies += @{
from = $programName
to = $calledProgram
type = "call"
}
}
}
# Extract copybooks
$copies = Get-CopyStatements -FilePath $cobolFile.FullName
foreach ($copybook in $copies) {
if ($copybook) {
$copybooks += @{
name = $copybook
used_by = $programName
}
$dependencies += @{
from = $programName
to = $copybook
type = "copy"
}
}
}
# Extract file references
$fileRefs = Get-FileReferences -FilePath $cobolFile.FullName
foreach ($fileRef in $fileRefs) {
if ($fileRef) {
$files += @{
name = $fileRef
used_by = $programName
}
$dependencies += @{
from = $programName
to = $fileRef
type = "file"
}
}
}
}
# Build final JSON with summary
$uniqueCopybooks = $copybooks | Select-Object -Property name -Unique
$uniqueFiles = $files | Select-Object -Property name -Unique
$result = @{
programs = $programs
dependencies = $dependencies
copybooks = $uniqueCopybooks
files = $uniqueFiles
summary = @{
total_programs = $programs.Count
total_dependencies = $dependencies.Count
total_copybooks = $uniqueCopybooks.Count
total_files = $uniqueFiles.Count
}
}
# Write to file
$result | ConvertTo-Json -Depth 10 | Set-Content -Path $OutputFile -Encoding UTF8
Write-Host ""
Write-Host "Analysis complete! Results written to: $OutputFile"
Write-Host ""
Write-Host "Summary:"
Write-Host " Programs: $($programs.Count)"
Write-Host " Dependencies: $($dependencies.Count)"
Write-Host " Copybooks: $($uniqueCopybooks.Count)"
Write-Host " Files: $($uniqueFiles.Count)"
#!/bin/bash
#
# Analyze dependencies between JCL jobs, programs, and datasets.
# This script scans a directory of JCL jobs and generates a dependency graph.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SOURCE_DIR="${1:-.}"
# Output file
OUTPUT_FILE="dependencies.json"
echo "Analyzing dependencies in: $SOURCE_DIR"
echo "Output will be written to: $OUTPUT_FILE"
# Initialize JSON structure
cat > "$OUTPUT_FILE" << 'EOF'
{
"programs": [],
"dependencies": [],
"copybooks": [],
"files": []
}
EOF
# Function to extract program name from COBOL file
extract_program_name() {
local file="$1"
grep -i "PROGRAM-ID" "$file" | head -1 | sed -E 's/.*PROGRAM-ID[. ]+([A-Za-z0-9-]+).*/\1/' || basename "$file" .cbl
}
# Function to extract CALL statements
extract_calls() {
local file="$1"
grep -i "CALL" "$file" | grep -oE "'[A-Z0-9-]+'" | tr -d "'" || true
}
# Function to extract COPY statements
extract_copies() {
local file="$1"
grep -i "COPY" "$file" | grep -oE 'COPY +[A-Z0-9-]+' | awk '{print $2}' | tr -d '.' || true
}
# Function to extract SELECT/ASSIGN file names
extract_files() {
local file="$1"
grep -i "SELECT" "$file" | grep -oE 'SELECT +[A-Z0-9-]+' | awk '{print $2}' || true
}
# Find all COBOL files
echo "Scanning for COBOL files..."
COBOL_FILES=$(find "$SOURCE_DIR" -type f \( -name "*.cbl" -o -name "*.CBL" -o -name "*.cob" -o -name "*.COB" \) 2>/dev/null || true)
if [ -z "$COBOL_FILES" ]; then
echo "No COBOL files found in $SOURCE_DIR"
exit 1
fi
# Create temporary files for collecting data
PROGRAMS_TEMP=$(mktemp)
DEPS_TEMP=$(mktemp)
COPYBOOKS_TEMP=$(mktemp)
FILES_TEMP=$(mktemp)
# Cleanup on exit
trap "rm -f $PROGRAMS_TEMP $DEPS_TEMP $COPYBOOKS_TEMP $FILES_TEMP" EXIT
# Process each COBOL file
echo "$COBOL_FILES" | while read -r cobol_file; do
[ -z "$cobol_file" ] && continue
echo "Processing: $cobol_file"
program_name=$(extract_program_name "$cobol_file")
# Add program to list
echo "{\"name\": \"$program_name\", \"file\": \"$cobol_file\"}" >> "$PROGRAMS_TEMP"
# Extract calls
extract_calls "$cobol_file" | while read -r called_program; do
[ -z "$called_program" ] && continue
echo "{\"from\": \"$program_name\", \"to\": \"$called_program\", \"type\": \"call\"}" >> "$DEPS_TEMP"
done
# Extract copybooks
extract_copies "$cobol_file" | while read -r copybook; do
[ -z "$copybook" ] && continue
echo "{\"name\": \"$copybook\", \"used_by\": \"$program_name\"}" >> "$COPYBOOKS_TEMP"
echo "{\"from\": \"$program_name\", \"to\": \"$copybook\", \"type\": \"copy\"}" >> "$DEPS_TEMP"
done
# Extract file references
extract_files "$cobol_file" | while read -r file_ref; do
[ -z "$file_ref" ] && continue
echo "{\"name\": \"$file_ref\", \"used_by\": \"$program_name\"}" >> "$FILES_TEMP"
echo "{\"from\": \"$program_name\", \"to\": \"$file_ref\", \"type\": \"file\"}" >> "$DEPS_TEMP"
done
done
# Build final JSON using jq if available, otherwise use basic concatenation
if command -v jq &> /dev/null; then
echo "Building dependency graph with jq..."
PROGRAMS_JSON=$(cat "$PROGRAMS_TEMP" | jq -s '.' 2>/dev/null || echo "[]")
DEPS_JSON=$(cat "$DEPS_TEMP" | jq -s '.' 2>/dev/null || echo "[]")
COPYBOOKS_JSON=$(cat "$COPYBOOKS_TEMP" | jq -s 'unique' 2>/dev/null || echo "[]")
FILES_JSON=$(cat "$FILES_TEMP" | jq -s 'unique' 2>/dev/null || echo "[]")
jq -n \
--argjson programs "$PROGRAMS_JSON" \
--argjson deps "$DEPS_JSON" \
--argjson copybooks "$COPYBOOKS_JSON" \
--argjson files "$FILES_JSON" \
'{
programs: $programs,
dependencies: $deps,
copybooks: $copybooks,
files: $files,
summary: {
total_programs: ($programs | length),
total_dependencies: ($deps | length),
total_copybooks: ($copybooks | length),
total_files: ($files | length)
}
}' > "$OUTPUT_FILE"
else
echo "jq not found, generating basic JSON..."
# Fallback without jq (less elegant but works)
echo "{" > "$OUTPUT_FILE"
echo " \"programs\": [" >> "$OUTPUT_FILE"
cat "$PROGRAMS_TEMP" | sed '$ ! s/$/,/' >> "$OUTPUT_FILE"
echo " ]," >> "$OUTPUT_FILE"
echo " \"dependencies\": [" >> "$OUTPUT_FILE"
cat "$DEPS_TEMP" | sed '$ ! s/$/,/' >> "$OUTPUT_FILE"
echo " ]," >> "$OUTPUT_FILE"
echo " \"copybooks\": [" >> "$OUTPUT_FILE"
cat "$COPYBOOKS_TEMP" | sed '$ ! s/$/,/' >> "$OUTPUT_FILE"
echo " ]," >> "$OUTPUT_FILE"
echo " \"files\": [" >> "$OUTPUT_FILE"
cat "$FILES_TEMP" | sed '$ ! s/$/,/' >> "$OUTPUT_FILE"
echo " ]" >> "$OUTPUT_FILE"
echo "}" >> "$OUTPUT_FILE"
fi
echo ""
echo "Analysis complete! Results written to: $OUTPUT_FILE"
echo ""
echo "Summary:"
echo " Programs: $(grep -c "\"name\"" "$PROGRAMS_TEMP" || echo 0)"
echo " Dependencies: $(wc -l < "$DEPS_TEMP" | xargs)"
echo " Copybooks: $(sort -u "$COPYBOOKS_TEMP" | wc -l | xargs)"
echo " Files: $(sort -u "$FILES_TEMP" | wc -l | xargs)"
#!/usr/bin/env python3
"""
Estimate migration complexity for COBOL programs.
Calculates a complexity score based on various factors:
- Lines of code
- Number of paragraphs
- Control flow complexity
- External dependencies
- File operations
- SQL operations
"""
import argparse
import json
import re
from pathlib import Path
from typing import Dict, Any
class ComplexityEstimator:
"""Estimate migration complexity for COBOL programs."""
# Complexity weights
WEIGHTS = {
'loc_per_point': 100, # Lines per complexity point
'paragraph_weight': 2, # Each paragraph adds complexity
'call_weight': 5, # Each external call
'file_weight': 3, # Each file operation
'sql_weight': 4, # Each SQL statement
'goto_weight': 10, # GO TO statements
'alter_weight': 15, # ALTER statements (very complex)
'perform_varying_weight': 3, # Complex loops
}
def __init__(self, cobol_file: Path):
self.cobol_file = cobol_file
self.content = cobol_file.read_text(encoding='utf-8', errors='ignore')
self.lines = [line for line in self.content.split('\n') if line.strip() and not line.strip().startswith('*')]
def estimate(self) -> Dict[str, Any]:
"""Calculate complexity estimate."""
metrics = self._collect_metrics()
score = self._calculate_score(metrics)
return {
'program': self.cobol_file.name,
'metrics': metrics,
'complexity_score': score,
'complexity_level': self._get_complexity_level(score),
'estimated_effort_days': self._estimate_effort(score),
'risk_factors': self._identify_risks(metrics)
}
def _collect_metrics(self) -> Dict[str, int]:
"""Collect various code metrics."""
return {
'total_lines': len(self.lines),
'paragraphs': self._count_paragraphs(),
'calls': self._count_pattern(r'CALL\s+'),
'files': self._count_pattern(r'SELECT\s+'),
'sql_statements': self._count_pattern(r'EXEC\s+SQL'),
'goto_statements': self._count_pattern(r'GO\s+TO\s+'),
'alter_statements': self._count_pattern(r'ALTER\s+'),
'perform_varying': self._count_pattern(r'PERFORM\s+.*VARYING'),
'copybooks': self._count_pattern(r'COPY\s+'),
'if_statements': self._count_pattern(r'IF\s+'),
'evaluate_statements': self._count_pattern(r'EVALUATE\s+'),
}
def _count_paragraphs(self) -> int:
"""Count paragraph definitions."""
count = 0
for line in self.lines:
# Paragraph name followed by period at start of line
if re.match(r'^[A-Z0-9\-]+\.\s*$', line.strip()):
count += 1
return count
def _count_pattern(self, pattern: str) -> int:
"""Count occurrences of a regex pattern."""
count = 0
for line in self.lines:
if re.search(pattern, line, re.IGNORECASE):
count += 1
return count
def _calculate_score(self, metrics: Dict[str, int]) -> int:
"""Calculate overall complexity score."""
score = 0
# Base score from lines of code
score += metrics['total_lines'] / self.WEIGHTS['loc_per_point']
# Add weighted factors
score += metrics['paragraphs'] * self.WEIGHTS['paragraph_weight']
score += metrics['calls'] * self.WEIGHTS['call_weight']
score += metrics['files'] * self.WEIGHTS['file_weight']
score += metrics['sql_statements'] * self.WEIGHTS['sql_weight']
score += metrics['goto_statements'] * self.WEIGHTS['goto_weight']
score += metrics['alter_statements'] * self.WEIGHTS['alter_weight']
score += metrics['perform_varying'] * self.WEIGHTS['perform_varying_weight']
return int(score)
def _get_complexity_level(self, score: int) -> str:
"""Categorize complexity level."""
if score < 20:
return 'Simple'
elif score < 50:
return 'Moderate'
elif score < 100:
return 'Complex'
else:
return 'Very Complex'
def _estimate_effort(self, score: int) -> float:
"""Estimate effort in person-days."""
# Rough estimate: 1 point = 0.5 days
# This should be calibrated based on team experience
base_days = score * 0.5
# Add overhead for testing and documentation (30%)
return round(base_days * 1.3, 1)
def _identify_risks(self, metrics: Dict[str, int]) -> list:
"""Identify risk factors."""
risks = []
if metrics['goto_statements'] > 0:
risks.append({
'type': 'control_flow',
'description': f"Contains {metrics['goto_statements']} GO TO statements - requires refactoring",
'severity': 'high'
})
if metrics['alter_statements'] > 0:
risks.append({
'type': 'control_flow',
'description': f"Contains {metrics['alter_statements']} ALTER statements - complex refactoring needed",
'severity': 'critical'
})
if metrics['sql_statements'] > 10:
risks.append({
'type': 'database',
'description': f"High number of SQL operations ({metrics['sql_statements']}) - requires careful transaction design",
'severity': 'medium'
})
if metrics['calls'] > 5:
risks.append({
'type': 'dependencies',
'description': f"Calls {metrics['calls']} external programs - coordinate migration with dependencies",
'severity': 'medium'
})
if metrics['total_lines'] > 1000:
risks.append({
'type': 'size',
'description': f"Large program ({metrics['total_lines']} lines) - consider splitting into multiple services",
'severity': 'medium'
})
if metrics['files'] > 5:
risks.append({
'type': 'io',
'description': f"Accesses {metrics['files']} files - design appropriate data access layer",
'severity': 'low'
})
return risks
def main():
parser = argparse.ArgumentParser(description='Estimate COBOL program migration complexity')
parser.add_argument('cobol_file', type=Path, help='Path to COBOL source file')
parser.add_argument('--detailed', action='store_true', help='Show detailed metrics')
parser.add_argument('--json', action='store_true', help='Output as JSON')
args = parser.parse_args()
if not args.cobol_file.exists():
print(f"Error: File not found: {args.cobol_file}")
return 1
estimator = ComplexityEstimator(args.cobol_file)
result = estimator.estimate()
if args.json:
print(json.dumps(result, indent=2))
else:
print(f"\nComplexity Analysis: {result['program']}")
print("=" * 60)
print(f"Complexity Level: {result['complexity_level']}")
print(f"Complexity Score: {result['complexity_score']}")
print(f"Estimated Effort: {result['estimated_effort_days']} person-days")
if args.detailed:
print("\nDetailed Metrics:")
print("-" * 60)
for key, value in result['metrics'].items():
print(f" {key.replace('_', ' ').title()}: {value}")
if result['risk_factors']:
print("\nRisk Factors:")
print("-" * 60)
for risk in result['risk_factors']:
severity_marker = {
'critical': '🔴',
'high': '🟠',
'medium': '🟡',
'low': '🟢'
}.get(risk['severity'], '⚪')
print(f" {severity_marker} [{risk['severity'].upper()}] {risk['description']}")
else:
print("\n✅ No significant risk factors identified")
print()
return 0
if __name__ == '__main__':
exit(main())
#!/usr/bin/env python3
"""
Extract structure from JCL (Job Control Language) files.
This script analyzes JCL source code and extracts:
- Job structure (JOB, EXEC, DD statements)
- Program execution steps
- Dataset definitions
- Conditional logic (COND)
- Job dependencies
- Workflow structure
"""
import argparse
import json
import re
from pathlib import Path
from typing import Dict, List, Any
class LegacyStructureExtractor:
"""Extract structural information from legacy source programs."""
def __init__(self, source_file: Path):
self.source_file = source_file
self.content = source_file.read_text(encoding='utf-8', errors='ignore')
self.lines = self.content.split('\n')
def extract(self) -> Dict[str, Any]:
"""Extract all structural information."""
return {
'program_name': self.extract_program_name(),
'divisions': self.extract_divisions(),
'working_storage': self.extract_working_storage(),
'file_definitions': self.extract_file_definitions(),
'paragraphs': self.extract_paragraphs(),
'calls': self.extract_calls(),
'copybooks': self.extract_copybooks(),
'sql_operations': self.extract_sql_operations(),
'statistics': self.calculate_statistics()
}
def extract_program_name(self) -> str:
"""Extract program name from PROGRAM-ID."""
for line in self.lines:
match = re.search(r'PROGRAM-ID\.\s+(\S+)', line, re.IGNORECASE)
if match:
return match.group(1).rstrip('.')
return self.source_file.stem
def extract_divisions(self) -> Dict[str, int]:
"""Find line numbers where divisions start."""
divisions = {}
for i, line in enumerate(self.lines):
for div_name in ['IDENTIFICATION', 'ENVIRONMENT', 'DATA', 'PROCEDURE']:
if re.search(rf'{div_name}\s+DIVISION', line, re.IGNORECASE):
divisions[div_name] = i + 1
return divisions
def extract_working_storage(self) -> List[Dict[str, Any]]:
"""Extract Working-Storage variables."""
variables = []
in_working_storage = False
for line in self.lines:
if re.search(r'WORKING-STORAGE\s+SECTION', line, re.IGNORECASE):
in_working_storage = True
continue
if re.search(r'(PROCEDURE\s+DIVISION|LINKAGE\s+SECTION)', line, re.IGNORECASE):
in_working_storage = False
if in_working_storage:
match = re.match(r'\s*(\d{2})\s+(\S+)\s+(.+)', line)
if match:
level, name, rest = match.groups()
pic_match = re.search(r'PIC\s+(\S+)', rest, re.IGNORECASE)
variables.append({
'level': int(level),
'name': name,
'picture': pic_match.group(1) if pic_match else None,
'line': self.lines.index(line) + 1
})
return variables
def extract_file_definitions(self) -> List[Dict[str, str]]:
"""Extract file definitions from SELECT statements."""
files = []
for line in self.lines:
match = re.search(r'SELECT\s+(\S+)\s+ASSIGN', line, re.IGNORECASE)
if match:
files.append({
'name': match.group(1),
'line': self.lines.index(line) + 1
})
return files
def extract_paragraphs(self) -> List[Dict[str, Any]]:
"""Extract paragraph and section names."""
paragraphs = []
for i, line in enumerate(self.lines):
# Match paragraph names (word followed by period at start of line)
match = re.match(r'^([A-Z0-9\-]+)\.\s*$', line.strip())
if match and i > 0:
para_name = match.group(1)
# Skip division names
if para_name not in ['IDENTIFICATION', 'ENVIRONMENT', 'DATA', 'PROCEDURE']:
paragraphs.append({
'name': para_name,
'line': i + 1,
'type': 'section' if 'SECTION' in para_name else 'paragraph'
})
return paragraphs
def extract_calls(self) -> List[Dict[str, Any]]:
"""Extract CALL statements to other programs."""
calls = []
for i, line in enumerate(self.lines):
match = re.search(r'CALL\s+[\'"](\S+)[\'"]', line, re.IGNORECASE)
if match:
calls.append({
'program': match.group(1),
'line': i + 1
})
return calls
def extract_copybooks(self) -> List[Dict[str, Any]]:
"""Extract COPY statements."""
copybooks = []
for i, line in enumerate(self.lines):
match = re.search(r'COPY\s+(\S+)', line, re.IGNORECASE)
if match:
copybooks.append({
'name': match.group(1).rstrip('.'),
'line': i + 1
})
return copybooks
def extract_sql_operations(self) -> List[Dict[str, Any]]:
"""Extract embedded SQL operations."""
sql_ops = []
in_sql = False
sql_text = []
sql_start = 0
for i, line in enumerate(self.lines):
if 'EXEC SQL' in line.upper():
in_sql = True
sql_start = i + 1
sql_text = []
if in_sql:
sql_text.append(line.strip())
if 'END-EXEC' in line.upper() and in_sql:
in_sql = False
sql_ops.append({
'statement': ' '.join(sql_text),
'line': sql_start
})
return sql_ops
def calculate_statistics(self) -> Dict[str, int]:
"""Calculate basic statistics."""
return {
'total_lines': len(self.lines),
'working_storage_vars': len(self.extract_working_storage()),
'paragraphs': len(self.extract_paragraphs()),
'calls': len(self.extract_calls()),
'copybooks': len(self.extract_copybooks()),
'sql_operations': len(self.extract_sql_operations())
}
def main():
parser = argparse.ArgumentParser(description='Extract structure from legacy source files')
parser.add_argument('source_file', type=Path, help='Path to source file')
parser.add_argument('--output', '-o', type=Path, help='Output JSON file (default: stdout)')
args = parser.parse_args()
if not args.source_file.exists():
print(f"Error: File not found: {args.source_file}")
return 1
extractor = LegacyStructureExtractor(args.source_file)
structure = extractor.extract()
output_json = json.dumps(structure, indent=2)
if args.output:
args.output.write_text(output_json)
print(f"Structure written to {args.output}")
else:
print(output_json)
return 0
if __name__ == '__main__':
exit(main())
#!/usr/bin/env python3
"""
Generate Java POJO classes from legacy data structures.
This script parses legacy data structure definitions and generates corresponding
Java classes with appropriate data types, getters, setters, and validation.
"""
import argparse
import re
from pathlib import Path
from typing import List, Dict, Optional, Tuple
class FieldDefinition:
"""Represents a COBOL field definition."""
def __init__(self, level: int, name: str, picture: Optional[str], occurs: Optional[int] = None):
self.level = level
self.name = name
self.picture = picture
self.occurs = occurs
self.children: List[FieldDefinition] = []
def is_group(self) -> bool:
"""Check if this is a group item (no picture clause)."""
return self.picture is None
def to_java_type(self) -> str:
"""Convert COBOL picture to Java type."""
if self.picture is None:
# Group item - will be a nested class
return self.to_java_class_name()
pic = self.picture.upper()
# Numeric types
if re.match(r'^S?9+V?9*$', pic.replace('(', '').replace(')', '')):
if 'V' in pic or 'COMP-3' in pic:
return 'BigDecimal'
digit_count = pic.count('9')
if digit_count <= 9:
return 'int' if pic.startswith('S') or pic.startswith('9') else 'long'
elif digit_count <= 18:
return 'long'
else:
return 'BigInteger'
# Alphanumeric
if pic.startswith('X'):
return 'String'
# Default to String for unknown patterns
return 'String'
def to_java_field_name(self) -> str:
"""Convert COBOL name to Java field name (camelCase)."""
parts = self.name.lower().replace('_', '-').split('-')
return parts[0] + ''.join(p.capitalize() for p in parts[1:])
def to_java_class_name(self) -> str:
"""Convert COBOL name to Java class name (PascalCase)."""
parts = self.name.lower().replace('_', '-').split('-')
return ''.join(p.capitalize() for p in parts)
class CopybookParser:
"""Parse COBOL copybook and extract field definitions."""
def __init__(self, copybook_file: Path):
self.copybook_file = copybook_file
self.content = copybook_file.read_text(encoding='utf-8', errors='ignore')
self.lines = self.content.split('\n')
def parse(self) -> FieldDefinition:
"""Parse copybook and return root field definition."""
fields = []
for line in self.lines:
# Match COBOL field definition: level number, name, and optional picture
match = re.match(r'\s*(\d{2})\s+(\S+)(?:\s+PIC\s+(\S+))?(?:\s+OCCURS\s+(\d+))?', line, re.IGNORECASE)
if match:
level = int(match.group(1))
name = match.group(2).rstrip('.')
picture = match.group(3)
occurs = int(match.group(4)) if match.group(4) else None
field = FieldDefinition(level, name, picture, occurs)
fields.append(field)
# Build hierarchy
return self._build_hierarchy(fields)
def _build_hierarchy(self, fields: List[FieldDefinition]) -> FieldDefinition:
"""Build hierarchical structure from flat field list."""
if not fields:
return FieldDefinition(1, 'Root', None)
root = fields[0]
stack = [root]
for field in fields[1:]:
# Pop stack until we find the parent level
while stack and stack[-1].level >= field.level:
stack.pop()
if stack:
stack[-1].children.append(field)
stack.append(field)
return root
class JavaClassGenerator:
"""Generate Java class from COBOL copybook structure."""
def __init__(self, root_field: FieldDefinition, package_name: str = 'com.example.model'):
self.root = root_field
self.package_name = package_name
self.imports = set()
def generate(self) -> str:
"""Generate complete Java class."""
self._collect_imports(self.root)
lines = []
lines.append(f'package {self.package_name};')
lines.append('')
# Add imports
if self.imports:
for imp in sorted(self.imports):
lines.append(f'import {imp};')
lines.append('')
# Generate class
lines.append('/**')
lines.append(f' * Generated from COBOL copybook: {self.root.name}')
lines.append(' * Auto-generated - do not modify directly')
lines.append(' */')
lines.extend(self._generate_class(self.root, 0))
return '\n'.join(lines)
def _collect_imports(self, field: FieldDefinition):
"""Collect required imports."""
java_type = field.to_java_type()
if java_type == 'BigDecimal':
self.imports.add('java.math.BigDecimal')
elif java_type == 'BigInteger':
self.imports.add('java.math.BigInteger')
elif field.occurs:
self.imports.add('java.util.List')
self.imports.add('java.util.ArrayList')
for child in field.children:
self._collect_imports(child)
def _generate_class(self, field: FieldDefinition, indent_level: int) -> List[str]:
"""Generate class definition recursively."""
lines = []
indent = ' ' * indent_level
class_name = field.to_java_class_name()
lines.append(f'{indent}public class {class_name} {{')
# Generate fields
for child in field.children:
field_indent = ' ' * (indent_level + 1)
java_type = child.to_java_type()
field_name = child.to_java_field_name()
if child.occurs:
lines.append(f'{field_indent}private List<{java_type}> {field_name} = new ArrayList<>();')
else:
lines.append(f'{field_indent}private {java_type} {field_name};')
if field.children:
lines.append('')
# Generate getters and setters
for child in field.children:
field_indent = ' ' * (indent_level + 1)
java_type = child.to_java_type()
field_name = child.to_java_field_name()
method_suffix = field_name[0].upper() + field_name[1:]
if child.occurs:
return_type = f'List<{java_type}>'
lines.append(f'{field_indent}public {return_type} get{method_suffix}() {{')
lines.append(f'{field_indent} return {field_name};')
lines.append(f'{field_indent}}}')
lines.append('')
lines.append(f'{field_indent}public void set{method_suffix}({return_type} {field_name}) {{')
lines.append(f'{field_indent} this.{field_name} = {field_name};')
lines.append(f'{field_indent}}}')
else:
lines.append(f'{field_indent}public {java_type} get{method_suffix}() {{')
lines.append(f'{field_indent} return {field_name};')
lines.append(f'{field_indent}}}')
lines.append('')
lines.append(f'{field_indent}public void set{method_suffix}({java_type} {field_name}) {{')
lines.append(f'{field_indent} this.{field_name} = {field_name};')
lines.append(f'{field_indent}}}')
lines.append('')
# Generate nested classes for group items
for child in field.children:
if child.is_group() and child.children:
lines.extend(self._generate_class(child, indent_level + 1))
lines.append('')
lines.append(f'{indent}}}')
return lines
def main():
parser = argparse.ArgumentParser(description='Generate Java classes from legacy data structures')
parser.add_argument('copybook', type=Path, help='Path to data structure definition file')
parser.add_argument('--package', '-p', default='com.example.model', help='Java package name')
parser.add_argument('--output-dir', '-o', type=Path, help='Output directory for Java files')
args = parser.parse_args()
if not args.copybook.exists():
print(f"Error: Data structure file not found: {args.copybook}")
return 1
# Parse copybook
parser_obj = CopybookParser(args.copybook)
root_field = parser_obj.parse()
# Generate Java class
generator = JavaClassGenerator(root_field, args.package)
java_code = generator.generate()
# Write output
if args.output_dir:
args.output_dir.mkdir(parents=True, exist_ok=True)
# Convert package name to path (e.g., com.example.model -> com/example/model)
package_parts = args.package.split('.')
package_path = args.output_dir.joinpath(*package_parts)
package_path.mkdir(parents=True, exist_ok=True)
output_file = package_path / f'{root_field.to_java_class_name()}.java'
output_file.write_text(java_code)
print(f"Generated: {output_file}")
else:
print(java_code)
return 0
if __name__ == '__main__':
exit(main())
Related skills
FAQ
Why is COND logic called critical?
The skill states COND is inverted and must be truth-tabled to avoid wrong step execution in migrated workflows.
Which scripts are bundled?
It references analyze-dependencies.sh, extract-structure.py, and related automation under scripts/.
What orchestration targets are listed?
It names Spring Batch, Apache Airflow, Kubernetes Jobs, shell scripts, AWS Step Functions, and Azure Logic Apps.