Spring Batch教程（六）spring boot实现batch功能注解示例：读文件写入mysql

spring batch spring boot 2023-08-20 11:08:34 717人浏览八月长安

摘要

spring batch 系列文章 Spring Batch教程（一）简单的介绍以及通过springbatch将xml文件转成txt文件 Spring Batch教程（二）示例：将txt文件转成xml文件以及读取xml文件内容存储到数据库

spring batch 系列文章

Spring Batch教程（一）简单的介绍以及通过springbatch将xml文件转成txt文件
 Spring Batch教程（二）示例：将txt文件转成xml文件以及读取xml文件内容存储到数据库mysql
Spring Batch教程（三）示例：从mysql中读取数据写入文本和从多个文本中读取内容写入mysql
Spring Batch教程（四）tasklet使用示例：spring batch的定时任务使用
 Spring Batch教程（五）spring boot实现batch功能注解示例：读写文本文件
 Spring Batch教程（六）spring boot实现batch功能注解示例：读文件写入mysql

本文是以注解的方式实现了读取文本数据，求和后存储至Mysql。
本文使用的是jdk8，较高版本的spring batch需要jdk11环境。

一、示例：读文本文件内容，求和计算后写入mysql

1、Maven依赖

<dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-batch</artifactId><version>2.3.12.RELEASE</version></dependency><dependency><groupId>org.projectlombok</groupId><artifactId>lombok</artifactId><version>1.18.2</version><scope>provided</scope></dependency><dependency><groupId>mysql</groupId><artifactId>mysql-connector-java</artifactId><version>5.1.31</version></dependency><dependency><groupId>com.mchange</groupId><artifactId>c3p0</artifactId><version>0.9.5-pre8</version></dependency>

2、java bean

1）、Student

import lombok.AllArgsConstructor;import lombok.Data;import lombok.NoArgsConstructor;@Data@AllArgsConstructor@NoArgsConstructorpublic class Student {private String id;private int chinese;private int math;private int english;}

2）、StudentTotalScore

import lombok.AllArgsConstructor;import lombok.Data;import lombok.NoArgsConstructor;@Data@AllArgsConstructor@NoArgsConstructorpublic class StudentTotalScore {private String id;private int totalScore;}

3、配置spring batch的ItemReader、ItemWriter和ItemProcessor

import java.beans.PropertyVetoException;import javax.sql.DataSource;import org.springframework.batch.core.Job;import org.springframework.batch.core.Step;import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;import org.springframework.batch.item.ItemProcessor;import org.springframework.batch.item.ItemReader;import org.springframework.batch.item.ItemWriter;import org.springframework.batch.item.database.BeanPropertyItemSqlParameterSourceProvider;import org.springframework.batch.item.database.JdbcBatchItemWriter;import org.springframework.batch.item.file.FlatFileItemReader;import org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper;import org.springframework.batch.item.file.mapping.DefaultLineMapper;import org.springframework.batch.item.file.transfORM.DelimitedLineTokenizer;import org.springframework.context.annotation.Bean;import org.springframework.context.annotation.Configuration;import org.springframework.core.io.ClassPathResource;import org.springframework.jdbc.core.JdbcTemplate;import com.mchange.v2.c3p0.ComboPooledDataSource;import com.win.writedbbyannotation.bean.Student;import com.win.writedbbyannotation.bean.StudentTotalScore;@Configuration@EnableBatchProcessingpublic class BatchConfiguration {private final String inputFiles = "student-data.txt";@Beanpublic ItemReader<Student> reader() {FlatFileItemReader<Student> reader = new FlatFileItemReader<Student>();reader.setResource(new ClassPathResource(inputFiles));reader.setLineMapper(new DefaultLineMapper<Student>() {{setLineTokenizer(new DelimitedLineTokenizer() {{setNames(new String[] { "id", "chinese", "math", "english" });}});setFieldSetMapper(new BeanWrapperFieldSetMapper<Student>() {{setTargetType(Student.class);}});}});return reader;}@Beanpublic ItemWriter<StudentTotalScore> writer(DataSource dataSource) {JdbcBatchItemWriter<StudentTotalScore> writer = new JdbcBatchItemWriter<StudentTotalScore>();writer.setItemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<StudentTotalScore>());writer.setSql("INSERT INTO studenttotalscore (id,totalscore) VALUES (:id,:totalScore)");writer.setDataSource(dataSource);return writer;}@Beanpublic ItemProcessor<Student, StudentTotalScore> processor() {return new StudentItemProcessor();}@Beanpublic Job createMarkSheet(JobBuilderFactory jobs, Step step) {return jobs.get("createMarkSheet").flow(step).end().build();}@Beanpublic Step step(StepBuilderFactory stepBuilderFactory, ItemReader<Student> reader, ItemWriter<StudentTotalScore> writer, ItemProcessor<Student, StudentTotalScore> processor) {return stepBuilderFactory.get("step").<Student, StudentTotalScore>chunk(5).reader(reader).processor(processor).writer(writer).build();}@Beanpublic JdbcTemplate jdbcTemplate(DataSource dataSource) {return new JdbcTemplate(dataSource);}@Beanpublic DataSource getDataSource() throws PropertyVetoException {ComboPooledDataSource dataSource = new ComboPooledDataSource();dataSource.setDriverClass("com.mysql.jdbc.Driver");dataSource.setJdbcUrl("jdbc:mysql://192.168.10.44:3306/test");dataSource.setUser("root");dataSource.setPassWord("12345");return dataSource;}}

4、创建ItemProcessor实现类

import org.springframework.batch.item.ItemProcessor;import com.win.writedbbyannotation.bean.Student;import com.win.writedbbyannotation.bean.StudentTotalScore;public class StudentItemProcessor implements ItemProcessor<Student, StudentTotalScore> {@Overridepublic StudentTotalScore process(final Student student) throws Exception {int totalScore = student.getChinese() + student.getMath() + student.getEnglish();System.out.println("student id:" + student.getId() + " and Total score:" + totalScore);StudentTotalScore studentTotalScore = new StudentTotalScore(student.getId(), totalScore);return studentTotalScore;}}

5、创建一个运行job的main类

import java.sql.ResultSet;import java.sql.SQLException;import java.util.List;import org.springframework.boot.SpringApplication;import org.springframework.boot.autoconfigure.EnableAutoConfiguration;import org.springframework.context.ApplicationContext;import org.springframework.context.annotation.ComponentScan;import org.springframework.jdbc.core.JdbcTemplate;import org.springframework.jdbc.core.RowMapper;import com.win.writedbbyannotation.bean.StudentTotalScore;@ComponentScan@EnableAutoConfiguration//@SpringBootApplicationpublic class App {public static void main(String[] args) {ApplicationContext ctx = SpringApplication.run(App.class, args);List result = ctx.getBean(JdbcTemplate.class).query("select id,totalscore FROM StudentTotalScore", new RowMapper() {@Overridepublic StudentTotalScore mapRow(ResultSet rs, int row) throws SQLException {return new StudentTotalScore(rs.getString(1), Integer.parseInt(rs.getString(2)));}});System.out.println("记录数:" + result.size());}}

6、准备测试数据

在工程项目中的resources文件夹下创建student-data.txt文件，并写入如下内容

student-1,90,85,96student-2,92,97,94student-3,95,93,100

7、创建数据库和表

SET NAMES utf8mb4;SET FOREIGN_KEY_CHECKS = 0;-- ------------------------------ Table structure for studenttotalscore-- ----------------------------DROP TABLE IF EXISTS `studenttotalscore`;CREATE TABLE `studenttotalscore`  (  `id` varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL,  `totalScore` int(11) NULL DEFAULT NULL) ENGINE = InnoDB CHARACTER SET = utf8 COLLATE = utf8_general_ci ROW_FORMAT = Dynamic;

8、验证

启动app.java，然后观察应用程序控制台与mysql数据库中数据的变化。
在启动应用程序时可能会出现如下异常：

Table 'XXX.BATCH_JOB_INSTANCE' doesn't exist

因为Spring Batch 会启用一个 H2 数据库，在这个数据库中，Sping 会对 Batch 需要的配置进行配置。如果使用 Spring JPA 的话，需要 Spring Batch 帮你初始化表。
解决办法：
application.properties文件中增加如下配置

spring.batch.initialize-schema=ALWAYS# 或  Spring Boot 2.7 的版本中spring.batch.jdbc.initialize-schema=ALWAYS

1）、应用程序输出结果

控制台部分输出

 Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.  NOT STARTED.  Currently in standby mode.  Number of jobs executed: 0  Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads.  Using job-store 'org.quartz.simpl.RAMJobStore' - which does not support persistence. and is not clustered.2023-07-24 11:08:33.785  INFO 100012 --- [           main] org.quartz.impl.StdSchedulerFactory      : Quartz scheduler 'quartzScheduler' initialized from an externally provided properties instance.2023-07-24 11:08:33.785  INFO 100012 --- [           main] org.quartz.impl.StdSchedulerFactory      : Quartz scheduler version: 2.2.12023-07-24 11:08:33.785  INFO 100012 --- [           main] org.quartz.core.QuartzScheduler          : JobFactory set to: org.springframework.scheduling.quartz.SpringBeanJobFactory@73D60e762023-07-24 11:08:33.804  INFO 100012 --- [           main] o.s.s.quartz.SchedulerFactoryBean        : Starting Quartz Scheduler now2023-07-24 11:08:33.804  INFO 100012 --- [           main] org.quartz.core.QuartzScheduler          : Scheduler quartzScheduler_$_NON_CLUSTERED started.2023-07-24 11:08:33.812  INFO 100012 --- [           main] com.win.writedbbyannotation.App          : Started App in 1.949 seconds (JVM running for 2.226)2023-07-24 11:08:33.814  INFO 100012 --- [           main] o.s.b.a.b.JobLauncherApplicationRunner   : Running default command line with: []2023-07-24 11:08:33.936  INFO 100012 --- [           main] o.s.b.c.l.support.SimpleJobLauncher      : Job: [FlowJob: [name=createMarkSheet]] launched with the following parameters: [{}]2023-07-24 11:08:33.971  INFO 100012 --- [           main] o.s.batch.core.job.SimpleStepHandler     : Step already complete or not restartable, so no action to execute: StepExecution: id=1, version=3, name=step, status=COMPLETED, exitStatus=COMPLETED, readCount=3, filterCount=0, writeCount=3 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0, exitDescription=2023-07-24 11:08:33.979  INFO 100012 --- [           main] o.s.b.c.l.support.SimpleJobLauncher      : Job: [FlowJob: [name=createMarkSheet]] completed with the following parameters: [{}] and the following status: [COMPLETED] in 21ms记录数:3