襄阳网站开发,平面设计培训班价格,国内课程网站建设现状,s什么网站可以接单做设计赚钱文章目录 day4学习内容自媒体文章自动审核今日内容 1 自媒体文章自动审核1.1 审核流程1.2 内容安全第三方接口1.3 引入阿里云内容安全接口1.3.1 添加依赖1.3.2 导入aliyun模块1.3.3 注入Bean测试 2 app端文章保存接口2.1 表结构说明2.2 分布式id2.2.1 分布式id-技术选型2.2.2 雪… 文章目录 day4学习内容自媒体文章自动审核今日内容 1 自媒体文章自动审核1.1 审核流程1.2 内容安全第三方接口1.3 引入阿里云内容安全接口1.3.1 添加依赖1.3.2 导入aliyun模块1.3.3 注入Bean测试 2 app端文章保存接口2.1 表结构说明2.2 分布式id2.2.1 分布式id-技术选型2.2.2 雪花算法2.2.3 配置雪花算法 2.3 保存app端文章-思路分析2.4 实现接口2.4.1 实现步骤2.4.2 定义feign接口2.4.2.1 导入feign远程调用依赖2.4.2.2 定义文章端远程接口2.4.2.3 导入ArticleDto 2.4.3 实现feign接口2.4.4 创建mapper2.4.5 为AparticleConfig设置默认参数2.4.6 在ApArticleService的实现类ApArticleServiceImpl中实现方法2.4.7 启动ArticleApplication 3 自媒体文章审核实现3.1 创建审核接口3.2 实现审核接口3.3 启动类扫描feign3.4 测试 4 自媒体调用文章微服务feign远程调用服务降级4.1 feign远程调用服务降级处理的逻辑4.2 编写降级逻辑4.3 指定IArticleClient接口指向Feign降级逻辑4.4 加载feign降级逻辑4.5 配置降级策略4.6 测试 5 文章审核异步调用5.1 在自动审核的方法加上Async注解5.2 在文章发布后调用自动审核方法5.3 在启动类中添加注解开启异步调用5.4 综合测试5.5 使用rabbit MQ来完成异步调用5.5.1 引入依赖5.5.2 为微服务配置MQ5.5.3 改造方法创建监听队列5.5.4 序列化MQ消息5.5.5 加上mq后的综合测试 6 自管理敏感词过滤6.1 DFA实现原理6.2 DFA检索过程6.3 实现步骤6.3.1 创建敏感词表6.3.2 将wm_sensitive对应的实体类和mapper导入6.3.3 在阿里云接口前自行进行审查6.3.4 测试 7 图片文字敏感词过滤7.1 文字图片识别7.2 Tesseract-OCR7.3 Tess4j案例7.3.1 导入依赖7.3.2 将训练好的分类器放入资源中7.3.3 demo7.3.4 结果 7.4 图片文字敏感词过滤实现7.4.1 创建工具类7.4.2 工具类被其他微服务使用7.4.3 在微服务中配置7.4.4 添加实现 8 静态文件生成8.1 实现思路8.1.1 生成minio接口和实现并且异步调用8.1.2 修改saveArticle逻辑8.1.3 开启异步调用8.1.4 测试 day4学习内容
自媒体文章自动审核 今日内容 1 自媒体文章自动审核
1.1 审核流程 1.2 内容安全第三方接口 1.3 引入阿里云内容安全接口 1.3.1 添加依赖
在heima-leadnews-common包下引入依赖
dependencygroupIdcom.aliyun/groupIdartifactIdaliyun-java-sdk-core/artifactIdversion4.1.1/version
/dependency
dependencygroupIdcom.aliyun/groupIdartifactIdaliyun-java-sdk-green/artifactIdversion3.6.6/version
/dependency
dependencygroupIdcom.alibaba.fastjson2/groupIdartifactIdfastjson2/artifactIdversion2.0.9/version
/dependency
dependencygroupIdcom.aliyun.oss/groupIdartifactIdaliyun-sdk-oss/artifactIdversion2.8.3/version
/dependency1.3.2 导入aliyun模块
放入heima-leadnews-common模块下的com.heima.common
哪个微服务使用就在哪个微服务的nacos中配置
在heima-leadnews-wemedia中的nacos配置中心添加以下配置
aliyun:accessKeyId: LTAI5tCWHCcfvqQzu8k2oKmXsecret: auoKUFsghimbfVQHpy7gtRyBkoR4vc
#aliyun.scenesporn,terrorism,ad,qrcode,live,logoscenes: terrorism1.3.3 注入Bean测试
在resource中META-INF的spring-factories中自动配置
org.springframework.boot.autoconfigure.EnableAutoConfiguration\com.heima.common.exception.ExceptionCatch,\com.heima.common.aliyun.GreenTextScan,\com.heima.common.aliyun.GreenImageScan在测试类中进行测试
SpringBootTest(classes WemediaApplication.class)
RunWith(SpringRunner.class)
public class AliyunTest {Autowiredprivate GreenTextScan greenTextScan;Autowiredprivate GreenImageScan greenImageScan;Autowiredprivate FileStorageService fileStorageService;Testpublic void testScanText() throws Exception {Map map greenTextScan.greeTextScan(我是一个好人,冰毒);System.out.println(map);}Testpublic void testScanImage() throws Exception {byte[] bytes fileStorageService.downLoadFile(http://192.168.200.130:9000/leadnews/2021/04/26/ef3cbe458db249f7bd6fb4339e593e55.jpg);Map map greenImageScan.imageScan(Arrays.asList(bytes));System.out.println(map);}
}2 app端文章保存接口
2.1 表结构说明 2.2 分布式id 2.2.1 分布式id-技术选型 2.2.2 雪花算法 2.2.3 配置雪花算法
第一在实体类中的id上加入如下配置指定类型为id_worker
TableId(value id,type IdType.ID_WORKER)
private Long id;第二在application.yml文件中配置数据中心id和机器id
在文章的微服务的nacos配置中leadnews-article中添加
spring:datasource:driver-class-name: com.mysql.jdbc.Driverurl: jdbc:mysql://localhost:3306/leadnews_article?useUnicodetruecharacterEncodingUTF-8serverTimezoneUTCuseSSLfalseusername: rootpassword: 123sjbsjb# 设置Mapper接口所对应的XML文件位置如果你在Mapper接口中有自定义方法需要进行该配置
mybatis-plus:mapper-locations: classpath*:mapper/*.xml# 设置别名包扫描路径通过该属性可以给包中的类注册别名type-aliases-package: com.heima.model.article.pojos#雪花算法global-config:datacenter-id: 1workerId: 1
minio:accessKey: miniosecretKey: minio123bucket: leadnewsendpoint: http://192.168.204.129:9000readPath: http://192.168.204.129:90002.3 保存app端文章-思路分析 2.4 实现接口 2.4.1 实现步骤 2.4.2 定义feign接口
2.4.2.1 导入feign远程调用依赖
在heima-leadnews-feign-api的pom.xml中导入依赖
dependencygroupIdorg.springframework.cloud/groupIdartifactIdspring-cloud-starter-openfeign/artifactId
/dependency2.4.2.2 定义文章端远程接口
heima-leadnews-feign-api定义com.heima.apis.article.IArticleClient接口
FeignClient(value leadnews-article)FeignClient指定文章远程调用接口名称
FeignClient(value leadnews-article)
public interface IArticleClient {PostMapping(/api/v1/article/save)public ResponseResult saveArticle(RequestBody ArticleDto dto) ;
}2.4.2.3 导入ArticleDto
在heima-leadnews-model模块下com.heima.model.article.dto中导入ArticleDto类
Data
public class ArticleDto extends ApArticle {/*** 文章内容*/private String content;
}2.4.3 实现feign接口
在heima-leadnews-service模块下的heima-leadnews-article模块下创建com.heima.article.feign.ArticleClient类
RestController
public class ArticleClient implements IArticleClient {Autowiredprivate ApArticleService apArticleService;PostMapping(/api/v1/article/save)Overridepublic ResponseResult saveArticle(RequestBody ArticleDto dto) {return apArticleService.saveArticle(dto);}
}2.4.4 创建mapper
在heima-leadnews-service模块下的heima-leadnews-article模块下创建com.heima.article.mapper.ApArticleConfigMapper接口
Mapper
public interface ApArticleConfigMapper extends BaseMapperApArticleConfig {
}2.4.5 为AparticleConfig设置默认参数
添加NoArgsConstructor
public ApArticleConfig(Long articleId) {this.articleId articleId;this.isDelete false;this.isDown false;this.isForward true;this.isComment true;
}添加有参构造
2.4.6 在ApArticleService的实现类ApArticleServiceImpl中实现方法
ApArticleService接口
public interface ApArticleService extends IServiceApArticle{/*** 加载文章列表* param dto* param type 1 加载更多 2 加载最新* return*/public ResponseResult load(ArticleHomeDto dto, Short type);/*** 保存文章* param dto* return*/public ResponseResult saveArticle(ArticleHomeDto dto);
}实现类实现saveArticle方法
Autowired
private ApArticleConfigMapper apArticleConfigMapper;
Autowired
private ApArticleContentMapper apArticleContentMapper;
/*** 保存文章* param dto* return*/
Override
public ResponseResult saveArticle(ArticleDto dto) {//1.参数检查if(dto null){return ResponseResult.errorResult(AppHttpCodeEnum.PARAM_INVALID);}ApArticle apArticle new ApArticle();//org.springframework.beansBeanUtils.copyProperties(dto, apArticle);//2.判断是否存在idif(dto.getId() null) {//2.1 不存在id 新增 文章、内容、配置save(apArticle);//2.1.2 保存文章配置ApArticleConfig apArticleConfig new ApArticleConfig(apArticle.getId());apArticleConfigMapper.insert(apArticleConfig);//2.1.3 保存文章内容ApArticleContent apArticleContent new ApArticleContent();apArticleContent.setArticleId(apArticle.getId());apArticleContent.setContent(dto.getContent());apArticleContentMapper.insert(apArticleContent);}else {//2.2 存在id更新 文章、内容//2.2.1 更新文章updateById(apArticle);//2.2.2 更新文章内容ApArticleContent apArticleContent apArticleContentMapper.selectOne(Wrappers.ApArticleContentlambdaQuery().eq(ApArticleContent::getArticleId, dto.getId()));apArticleContent.setContent(dto.getContent());apArticleContentMapper.updateById(apArticleContent);}//3.返回结果 文章的idreturn ResponseResult.okResult(apArticle.getId());
}2.4.7 启动ArticleApplication 刚刚是新增如果是修改。
就会在json中传入id 成功修改
3 自媒体文章审核实现
3.1 创建审核接口
在heima-leadnews-service中heima-leadnews-wemedia中的service新增WmNewAutoScanService接口
public interface WmNewAutoScanService {/*** 自动审核媒体文章*/public void autoScanMediaNews(Integer id);
}3.2 实现审核接口
Service
Slf4j
Transactional
public class WmNewAutoScanServiceImpl implements WmNewAutoScanService {Autowiredprivate WmNewsMapper wmNewsMapper;Qualifier(com.heima.apis.article.IArticleClient)Autowiredprivate IArticleClient iArticleClient;Autowiredprivate WmChannelMapper wmChannelMapper;Autowiredprivate WmUserMapper wmUserMapper;Overridepublic void autoScanMediaNews(Integer id) {//1.查询自媒体文章WmNews wmNews wmNewsMapper.selectById(id);if (wmNews null) {throw new RuntimeException(自媒体文章不存在);}if(wmNews.getStatus().equals(WmNews.Status.SUBMIT.getCode())){MapString,ListString scanMaterialsList extractImageAndContent(wmNews);//2.调用阿里云接口审核文本内容ListString contentTexts scanMaterialsList.get(contentTexts);boolean isTextScan true;if(!isTextScan)return;//3.调用阿里云接口审核图片内容ListString imagesUrls scanMaterialsList.get(imagesUrls);boolean isImageScan true;if(!isImageScan)return;if(isTextScan isImageScan) {//审核通过wmNews.setStatus((short) 9);wmNews.setReason(审核通过);}}//4.审核成功保存app端的相关文章数据ArticleDto dtonew ArticleDto();BeanUtils.copyProperties(wmNews,dto);//布局dto.setLayout(wmNews.getType());//频道dto.setChannelId(wmNews.getChannelId());//频道名称WmChannel wmChannel wmChannelMapper.selectById(wmNews.getChannelId());if(wmChannel!null){dto.setChannelName(wmChannel.getName());}//作者dto.setAuthorId(Long.valueOf(wmNews.getUserId()));//作者名称WmUser wmUser wmUserMapper.selectById(wmNews.getUserId());if(wmUser!null){dto.setAuthorName(wmUser.getName());}//设置文章idif(wmNews.getArticleId()!null){dto.setId(wmNews.getArticleId());}dto.setCreatedTime(new Date());ResponseResult responseResult iArticleClient.saveArticle(dto);if(responseResult.getCode().equals(200)){//保存成功wmNews.setArticleId((Long)responseResult.getData());wmNewsMapper.updateById(wmNews);}else{//保存失败throw new RuntimeException(保存app端文章失败);}}private MapString,ListString extractImageAndContent(WmNews wmNews) {//提取文章内容String content wmNews.getContent();ListString imagesUrls new ArrayList();ListString contentTexts new ArrayList();MapString,ListString scanMaterialsList new HashMap();ListMap maps JSON.parseArray(content, Map.class);//提取文章图片for (Map map : maps) {if(map.get(type).equals(image)){String imgUrl (String) map.get(value);imagesUrls.add(imgUrl);}if(map.get(type).equals(text)){String text (String) map.get(value);contentTexts.add(text);}}scanMaterialsList.put(imagesUrls,imagesUrls);scanMaterialsList.put(contentTexts,contentTexts);return scanMaterialsList;}
}3.3 启动类扫描feign
调用Feign远程接口时要在启动类中加入EnableFeignClients(basePackages “com.heima.apis”)来对feign的api进行扫描同时也要引入feign-api模块的依赖
dependencygroupIdcom.heima/groupIdartifactIdheima-leadnews-feign-api/artifactId
/dependencySpringBootApplication
EnableDiscoveryClient
MapperScan(com.heima.wemedia.mapper)
EnableFeignClients(basePackages com.heima.apis)
public class WemediaApplication {public static void main(String[] args) {SpringApplication.run(WemediaApplication.class,args);}Beanpublic MybatisPlusInterceptor mybatisPlusInterceptor() {MybatisPlusInterceptor interceptor new MybatisPlusInterceptor();interceptor.addInnerInterceptor(new PaginationInnerInterceptor(DbType.MYSQL));return interceptor;}
}3.4 测试
转到WmNewAutoScanService接口中ctrlshiftT创建测试类 SpringBootTest(classes WemediaApplication.class)
RunWith(SpringRunner.class)
class WmNewAutoScanServiceTest {Autowiredprivate WmNewAutoScanService wmNewAutoScanService;Testvoid autoScanMediaNews() {wmNewAutoScanService.autoScanMediaNews(6236);}
}4 自媒体调用文章微服务feign远程调用服务降级 4.1 feign远程调用服务降级处理的逻辑 4.2 编写降级逻辑
在heima-leadnews-feign-api模块下编写降级逻辑com.heima.apis.article.fallback.IArticleClientFallback类实现IArticleClient接口
Component
public class IArticleClientFallback implements IArticleClient {Overridepublic ResponseResult saveArticle(ArticleDto dto) {return ResponseResult.errorResult(AppHttpCodeEnum.SERVER_ERROR,获取数据失败);}
}4.3 指定IArticleClient接口指向Feign降级逻辑
FeignClient(value leadnews-article,fallback IArticleClientFallback.class)
FeignClient(value leadnews-article,fallback IArticleClientFallback.class)
public interface IArticleClient {PostMapping(/api/v1/article/save)public ResponseResult saveArticle(RequestBody ArticleDto dto) ;
}4.4 加载feign降级逻辑
因为IArticleClientFallback是在com.heima.apis.article.fallback包下并不能被spring通过Component直接加载
因此需要在使用的微服务中加载feign
如使用的微服务是heima-leadnews-wemedia所以要在com.heima.wemedia.config下创建InitConfig类加载feign降级策略
Configuration
ComponentScan(com.heima.apis.article.fallback)
public class InitConfig {
}4.5 配置降级策略
要么在bootstrap中开启要么在nacos中实现热更新
这里采用nacos热更新
feign:# 开启feign对hystrix熔断降级的支持hystrix:enabled: true# 修改调用超时时间client:config:default:connectTimeout: 2000readTimeout: 20004.6 测试
当前设置超时2s进行降级测试一下
在com.heima.article.service.impl.ApArticleServiceImpl类中的saveArticle方法添加睡眠3秒进行测试
Override
public ResponseResult saveArticle(ArticleDto dto) {try {Thread.sleep(3000);} catch (InterruptedException e) {throw new RuntimeException(e);}//1.参数检查if(dto null){这次审核6239
SpringBootTest(classes WemediaApplication.class)
RunWith(SpringRunner.class)
class WmNewAutoScanServiceTest {Autowiredprivate WmNewAutoScanService wmNewAutoScanService;Testvoid autoScanMediaNews() {wmNewAutoScanService.autoScanMediaNews(6239);}
}5 文章审核异步调用 5.1 在自动审核的方法加上Async注解
Springboot集成异步线程调用
Override
Async//表明这是一个异步方法
public void autoScanMediaNews(Integer id) {try {Thread.sleep(1000);} catch (InterruptedException e) {throw new RuntimeException(e);}5.2 在文章发布后调用自动审核方法
//5.审核文章
wmNewAutoScanService.autoScanMediaNews(wmNews.getId());Autowired
private WmNewAutoScanService wmNewAutoScanService;
Override
public ResponseResult submitNews(WmNewsDto wmNewsDto) {// 0.参数检查if(wmNewsDto null||wmNewsDto.getContent()null){return ResponseResult.errorResult(AppHttpCodeEnum.PARAM_INVALID);}//1. 保存或修改文章WmNews wmNews new WmNews();BeanUtils.copyProperties(wmNewsDto,wmNews);//1.1 封面if(wmNewsDto.getImages()!null wmNewsDto.getImages().size()0){String imageStr StringUtils.join(wmNewsDto.getImages(), ,);wmNews.setImages(imageStr);}//1.2 如果封面为自动-1则需要手动设置封面规则if(wmNewsDto.getType().equals(WemediaConstants.WM_NEWS_TYPE_AUTO)){wmNews.setType(null);}saveOrUpdateWmNews(wmNews);//2.判断是否为草稿如果为草稿结束当前方法if(wmNews.getStatus().equals(WmNews.Status.NORMAL.getCode())){return ResponseResult.okResult(AppHttpCodeEnum.SUCCESS);}//3.不是草稿保存文章内容与图片素材的关系//3.1 获取文章内容的图片素材ListString imageListextractUrlInfo(wmNewsDto.getContent());saveRelativeInfoForContent(imageList,wmNews.getId());//4.不是草稿保存文章封面图片与图片素材的关系saveRelativeInfoForCover(wmNewsDto,wmNews,imageList);//5.审核文章wmNewAutoScanService.autoScanMediaNews(wmNews.getId());return ResponseResult.okResult(AppHttpCodeEnum.SUCCESS);
}5.3 在启动类中添加注解开启异步调用
在自媒体引导类中使用EnableAsync注解开启异步调用
SpringBootApplication
EnableDiscoveryClient
MapperScan(com.heima.wemedia.mapper)
EnableFeignClients(basePackages com.heima.apis)
EnableAsync//开启异步
public class WemediaApplication {public static void main(String[] args) {SpringApplication.run(WemediaApplication.class,args);}Beanpublic MybatisPlusInterceptor mybatisPlusInterceptor() {MybatisPlusInterceptor interceptor new MybatisPlusInterceptor();interceptor.addInnerInterceptor(new PaginationInnerInterceptor(DbType.MYSQL));return interceptor;}
}5.4 综合测试 5.5 使用rabbit MQ来完成异步调用
我的异步调用只要在启动类中加入EnableAsync就报错迫不得已采用rabbitMQ
5.5.1 引入依赖
在heima-leadnews-service中引入依赖
dependencygroupIdorg.springframework.boot/groupIdartifactIdspring-boot-starter-amqp/artifactId
/dependency
dependencygroupIdcom.fasterxml.jackson.core/groupIdartifactIdjackson-databind/artifactId
/dependency5.5.2 为微服务配置MQ
在heima-leadnews-article和wemedia的配置文件中添加配置
spring:rabbitmq:host: 192.168.204.129port: 5672virtual-host: /username: itcastpassword: 1233215.5.3 改造方法创建监听队列
修改heima-leadnews-wemedia下的com.heima.wemedia.service.impl.WmNewAutoScanServiceImpl类中的autoScanMediaNews方法
Autowired
private RabbitTemplate rabbitTemplate;
Override
public void autoScanMediaNews(Integer id) {//1.查询自媒体文章WmNews wmNews wmNewsMapper.selectById(id);if (wmNews null) {throw new RuntimeException(自媒体文章不存在);}if(wmNews.getStatus().equals(WmNews.Status.SUBMIT.getCode())){MapString,ListString scanMaterialsList extractImageAndContent(wmNews);//2.调用阿里云接口审核文本内容ListString contentTexts scanMaterialsList.get(contentTexts);boolean isTextScan true;if(!isTextScan)return;//3.调用阿里云接口审核图片内容ListString imagesUrls scanMaterialsList.get(imagesUrls);boolean isImageScan true;if(!isImageScan)return;if(isTextScan isImageScan) {//审核通过wmNews.setStatus((short) 9);wmNews.setReason(审核通过);}}//4.审核成功保存app端的相关文章数据ArticleDto dtonew ArticleDto();BeanUtils.copyProperties(wmNews,dto);//布局dto.setLayout(wmNews.getType());//频道dto.setChannelId(wmNews.getChannelId());//频道名称WmChannel wmChannel wmChannelMapper.selectById(wmNews.getChannelId());if(wmChannel!null){dto.setChannelName(wmChannel.getName());}//作者dto.setAuthorId(Long.valueOf(wmNews.getUserId()));//作者名称WmUser wmUser wmUserMapper.selectById(wmNews.getUserId());if(wmUser!null){dto.setAuthorName(wmUser.getName());}//设置文章idif(wmNews.getArticleId()!null){dto.setId(wmNews.getArticleId());}dto.setCreatedTime(new Date());//2.rabbitmq异步处理MapString,Object mapnew HashMap();map.put(dto,dto);map.put(wmNewsId,id);rabbitTemplate.convertAndSend(article.queue, map);/*ResponseResult responseResult iArticleClient.saveArticle(dto);if(responseResult.getCode().equals(200)){//保存成功wmNews.setArticleId((Long)responseResult.getData());wmNewsMapper.updateById(wmNews);}else{//保存失败log.error(保存app端文章失败responseResult: {}, responseResult);throw new RuntimeException(保存app端文章失败);}*/
}rabbitTemplate.convertAndSend(article.queue, map);发送到article.queue队列
在heima-leadnews-article模块下创建com.heima.article.mq.ArticleMessageConsumer消费者监听类监听article.queue
Slf4j
Component
public class ArticleMessageConsumer {Autowiredprivate IArticleClient iArticleClient;Autowiredprivate RabbitTemplate rabbitTemplate;RabbitListener(bindings QueueBinding(valueQueue(namearticle.queue),exchangeExchange(namearticle.direct,type ExchangeTypes.FANOUT)))public void processMessage(MapString,Object map) {ObjectMapper objectMapper new ObjectMapper();Object dto map.get(dto);Integer id (Integer) map.get(wmNewsId);LinkedHashMapString, Object linkedHashMap (LinkedHashMapString, Object) dto;ArticleDto articleDto objectMapper.convertValue(linkedHashMap, ArticleDto.class);// 异步处理文章数据ResponseResult responseResult iArticleClient.saveArticle(articleDto);if(responseResult.getCode().equals(200)){WmNews wmNews new WmNews();BeanUtils.copyProperties(dto, wmNews);wmNews.setArticleId((Long)responseResult.getData());MapString,Object params new HashMap();params.put(id, id);params.put(wmNews, wmNews);params.put(articleId,(Long)responseResult.getData());rabbitTemplate.convertAndSend(wmNews.queue, params);log.info(发送params成功param: {}, params);}else{//保存失败log.error(保存app端文章失败responseResult: {}, responseResult);throw new RuntimeException(保存app端文章失败);}}
}将ResponseResult responseResult iArticleClient.saveArticle(articleDto);回填的id发到wmNews.queue
在heima-leadnews-wemedia模块下创建com.heima.wemedia.mq.ReceiveWmNewsId消费者监听类监听wmNews.queue
Component
Slf4j
public class ReceiveWmNewsId {Autowiredprivate WmNewsMapper wmNewsMapper;RabbitListener(bindings QueueBinding(valueQueue(namewmNews.queue),exchangeExchange(namewmNews.direct,type ExchangeTypes.FANOUT)))public void processMessage(MapString,Object map) {ObjectMapper objectMapper new ObjectMapper();Integer id (Integer)map.get(id);Object wmNews map.get(wmNews);Long articleId (Long)map.get(articleId);LinkedHashMapString, Object linkedHashMap (LinkedHashMapString, Object) wmNews;WmNews articleDto objectMapper.convertValue(linkedHashMap, WmNews.class);WmNews oldwmNews wmNewsMapper.selectById(id);BeanUtils.copyProperties(wmNews,oldwmNews);oldwmNews.setStatus((short) 9);oldwmNews.setReason(审核通过);oldwmNews.setArticleId(articleId);int i wmNewsMapper.updateById(oldwmNews);if(i 0){log.error(更新自媒体文章失败wmNews: {}, oldwmNews);throw new RuntimeException(更新自媒体文章失败);}}
}5.5.4 序列化MQ消息
在heima-leadnews-article和wemedia的启动类中添加序列化器
SpringBootApplication
EnableDiscoveryClient
MapperScan(com.heima.wemedia.mapper)
EnableFeignClients(basePackages com.heima.apis)
public class WemediaApplication {public static void main(String[] args) {SpringApplication.run(WemediaApplication.class,args);}Beanpublic MybatisPlusInterceptor mybatisPlusInterceptor() {MybatisPlusInterceptor interceptor new MybatisPlusInterceptor();interceptor.addInnerInterceptor(new PaginationInnerInterceptor(DbType.MYSQL));return interceptor;}Beanpublic MessageConverter messageConverter(){return new Jackson2JsonMessageConverter();}Beanpublic RabbitTemplate rabbitTemplate(ConnectionFactory connectionFactory, MessageConverter messageConverter) {RabbitTemplate rabbitTemplate new RabbitTemplate(connectionFactory);rabbitTemplate.setMessageConverter(messageConverter);return rabbitTemplate;}
}SpringBootApplication
EnableDiscoveryClient
MapperScan(com.heima.article.mapper)
public class ArticleApplication {public static void main(String[] args) {SpringApplication.run(ArticleApplication.class,args);}Beanpublic MybatisPlusInterceptor mybatisPlusInterceptor() {MybatisPlusInterceptor interceptor new MybatisPlusInterceptor();interceptor.addInnerInterceptor(new PaginationInnerInterceptor(DbType.MYSQL));return interceptor;}Beanpublic MessageConverter messageConverter(){return new Jackson2JsonMessageConverter();}Beanpublic RabbitTemplate rabbitTemplate(ConnectionFactory connectionFactory, MessageConverter messageConverter) {RabbitTemplate rabbitTemplate new RabbitTemplate(connectionFactory);rabbitTemplate.setMessageConverter(messageConverter);return rabbitTemplate;}
}5.5.5 加上mq后的综合测试 测试通过在MQ上也检测到消息 6 自管理敏感词过滤 6.1 DFA实现原理 6.2 DFA检索过程 6.3 实现步骤 6.3.1 创建敏感词表
在leadnews-wemedia数据库中到入wm_sensitive.sql
6.3.2 将wm_sensitive对应的实体类和mapper导入
Data
TableName(wm_sensitive)
public class WmSensitive implements Serializable {private static final long serialVersionUID 1L;/*** 主键*/TableId(value id, type IdType.AUTO)private Integer id;/*** 敏感词*/TableField(sensitives)private String sensitives;/*** 创建时间*/TableField(created_time)private Date createdTime;
}Mapper
public interface WmSensitiveMapper extends BaseMapperWmSensitive {
}6.3.3 在阿里云接口前自行进行审查
boolean isSensitive handleSensitiveWords(contentTexts,wmNews);
if(wmNews.getStatus().equals(WmNews.Status.SUBMIT.getCode())){MapString,ListString scanMaterialsList extractImageAndContent(wmNews);//2.调用阿里云接口审核文本内容ListString contentTexts scanMaterialsList.get(contentTexts);//2.1 敏感词过滤boolean isSensitive handleSensitiveWords(contentTexts,wmNews);boolean isTextScan true;if(!isTextScan)return;//3.调用阿里云接口审核图片内容ListString imagesUrls scanMaterialsList.get(imagesUrls);boolean isImageScan true;Autowired
private WmSensitiveMapper wmSensitiveMapper;
private boolean handleSensitiveWords(ListString contentTexts, WmNews wmNews) {boolean isSensitive true;//1.获取所有敏感词ListWmSensitive wmSensitiveList wmSensitiveMapper.selectList(Wrappers.WmSensitivelambdaQuery().select(WmSensitive::getSensitives));ListString collect wmSensitiveList.stream().map(WmSensitive::getSensitives).collect(Collectors.toList());//2.初始化敏感词库SensitiveWordUtil.initMap(collect);//3.遍历文章内容查看是否包含敏感词for(String contentText:contentTexts){MapString, Integer map SensitiveWordUtil.matchWords(contentText);if(map.size()0){//4.如果包含敏感词修改文章状态wmNews.setStatus((short) 2);wmNews.setReason(文章内容包含敏感词);wmNewsMapper.updateById(wmNews);isSensitive false;break;}}return isSensitive;
}6.3.4 测试 7 图片文字敏感词过滤
7.1 文字图片识别 7.2 Tesseract-OCR 7.3 Tess4j案例 7.3.1 导入依赖
在heima-leadnews-test模块下的tess4j-demo的模块下导入依赖
dependencygroupIdnet.sourceforge.tess4j/groupIdartifactIdtess4j/artifactIdversion4.1.1/version
/dependency7.3.2 将训练好的分类器放入资源中 7.3.3 demo
在tess4j-demo的Applcation中
public class Application {/*** 识别图片中的文字* param args*/public static void main(String[] args) {// 1.创建Tesseract对象Tesseract tesseract new Tesseract();// 2.设置训练库的位置tesseract.setDatapath(D:\\Code\\JavaCode\\HeimaToutiao\\heima-leadnews\\heima-leadnews-test\\tess4j-demo\\src\\main\\resources\\tessdata);// 3.设置识别语言tesseract.setLanguage(chi_sim);// 4.设置识别图片File file new File(D:\\Code\\JavaCode\\HeimaToutiao\\heima-leadnews\\heima-leadnews-test\\tess4j-demo\\src\\main\\resources\\testdata\\testImage.png);// 5.识别图片try {String result tesseract.doOCR(file);System.out.println(result.replace(\\n|\\r, ));} catch (TesseractException e) {e.printStackTrace();}}
}7.3.4 结果 7.4 图片文字敏感词过滤实现 7.4.1 创建工具类
在heima-leadnews-common中创建com.heima.common.tess4j.Tess4jClient工具类封装tess4j
Getter
Setter
Component
ConfigurationProperties(prefix tess4j)
public class Tess4jClient {private String dataPath;private String language;public String doOCR(BufferedImage image) throws TesseractException {//创建Tesseract对象ITesseract tesseract new Tesseract();//设置字体库路径tesseract.setDatapath(dataPath);//中文识别tesseract.setLanguage(language);//执行ocr识别String result tesseract.doOCR(image);//替换回车和tal键 使结果为一行result result.replaceAll(\\r|\\n, -).replaceAll( , );return result;}}7.4.2 工具类被其他微服务使用
想让工具类被其他微服务使用就要拷贝全路径在当前的resource中的META-INF的spring.factories中添加配置
org.springframework.boot.autoconfigure.EnableAutoConfiguration\com.heima.common.exception.ExceptionCatch,\com.heima.common.aliyun.GreenTextScan,\com.heima.common.aliyun.GreenImageScan,\com.heima.common.tess4j.Tess4jClient7.4.3 在微服务中配置
在heima-leadnews-wemedia中的resource的boostrap.yml中进行配置
tess4j:data-path: D:\Code\JavaCode\HeimaToutiao\heima-leadnews\heima-leadnews-test\tess4j-demo\src\main\resources\tessdatalanguage: chi_sim7.4.4 添加实现
在WmNewsAutoScanServiceImpl中的handleImageScan方法上添加如下代码
try {for (String image : images) {byte[] bytes fileStorageService.downLoadFile(image);//图片识别文字审核---begin-----//从byte[]转换为butteredImageByteArrayInputStream in new ByteArrayInputStream(bytes);BufferedImage imageFile ImageIO.read(in);//识别图片的文字String result tess4jClient.doOCR(imageFile);//审核是否包含自管理的敏感词boolean isSensitive handleSensitiveScan(result, wmNews);if(!isSensitive){return isSensitive;}//图片识别文字审核---end-----imageList.add(bytes);}
}catch (Exception e){e.printStackTrace();
}8 静态文件生成 8.1 实现思路
我们在保存/修改文章时就应该同时异步的的生成静态文件生成静态文件上传到minio中
8.1.1 生成minio接口和实现并且异步调用
在com.heima.article.service.ArticleFreemarkerService接口
生成静态文件上传到minio中
public interface ArticleFreemarkerService {/*** 生成静态化页面* param apArticle* param content*/public void buildArticleToMinio(ApArticle apArticle,String content);
}Service
Slf4j
Transactional
public class ArticleFreemarkerServiceImpl implements ArticleFreemarkerService {Autowiredprivate ApArticleContentMapper apArticleContentMapper;Autowiredprivate Configuration configuration;Autowiredprivate FileStorageService fileStorageService;Autowiredprivate ApArticleService apArticleService;/*** 生成静态化页面* param apArticle* param content*/AsyncOverridepublic void buildArticleToMinio(ApArticle apArticle, String content) {if(StringUtils.isNotBlank(content)){//1.文章内容通过freemarker生成静态html页面Template template null;//2 输出流StringWriter writer new StringWriter();try {template configuration.getTemplate(article.ftl);//2.1 创建模型MapString,Object contentDataModelnew HashMap();//content是固定的因为article.ftl中有#if content??${content}/#if//因为apArticleContent.getContent()获取的是字符串所以需要转换成对象contentDataModel.put(content, JSONArray.parseArray(content));//2.2 合成方法template.process(contentDataModel,writer);} catch (Exception e) {throw new RuntimeException(e);}//3.把静态页面上传到minio//3.1 文件流InputStream inputStream new ByteArrayInputStream(writer.toString().getBytes());String path fileStorageService.uploadHtmlFile(,apArticle.getId().html,inputStream);//4.把静态页面的路径保存到数据库apArticleService.update(Wrappers.ApArticlelambdaUpdate().eq(ApArticle::getId,apArticle.getId()).set(ApArticle::getStaticUrl,path));}}
}8.1.2 修改saveArticle逻辑
修改com.heima.article.service.impl.ApArticleServiceImpl的saveArticle方法添加buildArticleToMinio
articleFreemarkerService.buildArticleToMinio(apArticle, dto.getContent());Autowiredprivate ApArticleConfigMapper apArticleConfigMapper;Autowiredprivate ApArticleContentMapper apArticleContentMapper;Autowiredprivate ArticleFreemarkerService articleFreemarkerService;/*** 保存文章* param dto* return*/Overridepublic ResponseResult saveArticle(ArticleDto dto) {//1.参数检查if(dto null){return ResponseResult.errorResult(AppHttpCodeEnum.PARAM_INVALID);}ApArticle apArticle new ApArticle();//org.springframework.beansBeanUtils.copyProperties(dto, apArticle);//2.判断是否存在idif(dto.getId() null) {//2.1 不存在id 新增 文章、内容、配置save(apArticle);//2.1.2 保存文章配置ApArticleConfig apArticleConfig new ApArticleConfig(apArticle.getId());apArticleConfigMapper.insert(apArticleConfig);//2.1.3 保存文章内容ApArticleContent apArticleContent new ApArticleContent();apArticleContent.setArticleId(apArticle.getId());apArticleContent.setContent(dto.getContent());apArticleContentMapper.insert(apArticleContent);}else {//2.2 存在id更新 文章、内容//2.2.1 更新文章updateById(apArticle);//2.2.2 更新文章内容ApArticleContent apArticleContent apArticleContentMapper.selectOne(Wrappers.ApArticleContentlambdaQuery().eq(ApArticleContent::getArticleId, dto.getId()));apArticleContent.setContent(dto.getContent());apArticleContentMapper.updateById(apArticleContent);}//异步调用 生成静态文件上传到minio中articleFreemarkerService.buildArticleToMinio(apArticle, dto.getContent());//3.返回结果 文章的idreturn ResponseResult.okResult(apArticle.getId());}
}8.1.3 开启异步调用
引导类加上EnableAsyn
SpringBootApplication
EnableDiscoveryClient
MapperScan(com.heima.article.mapper)
EnableAsync
public class ArticleApplication {public static void main(String[] args) {SpringApplication.run(ArticleApplication.class,args);}Beanpublic MybatisPlusInterceptor mybatisPlusInterceptor() {MybatisPlusInterceptor interceptor new MybatisPlusInterceptor();interceptor.addInnerInterceptor(new PaginationInnerInterceptor(DbType.MYSQL));return interceptor;}Beanpublic MessageConverter messageConverter(){return new Jackson2JsonMessageConverter();}Beanpublic RabbitTemplate rabbitTemplate(ConnectionFactory connectionFactory, MessageConverter messageConverter) {RabbitTemplate rabbitTemplate new RabbitTemplate(connectionFactory);rabbitTemplate.setMessageConverter(messageConverter);return rabbitTemplate;}
}8.1.4 测试 查看minio有没有生成 生成成功查看数据库有html生成说明功能成功
文章转载自： http://www.morning.tnzwm.cn.gov.cn.tnzwm.cn http://www.morning.nllst.cn.gov.cn.nllst.cn http://www.morning.srgsb.cn.gov.cn.srgsb.cn http://www.morning.ktcrr.cn.gov.cn.ktcrr.cn http://www.morning.qdbcd.cn.gov.cn.qdbcd.cn http://www.morning.zttjs.cn.gov.cn.zttjs.cn http://www.morning.ttfh.cn.gov.cn.ttfh.cn http://www.morning.khlxd.cn.gov.cn.khlxd.cn http://www.morning.wjrq.cn.gov.cn.wjrq.cn http://www.morning.pjjkz.cn.gov.cn.pjjkz.cn http://www.morning.zlhcw.cn.gov.cn.zlhcw.cn http://www.morning.nzms.cn.gov.cn.nzms.cn http://www.morning.jcfqg.cn.gov.cn.jcfqg.cn http://www.morning.fgxnb.cn.gov.cn.fgxnb.cn http://www.morning.krswn.cn.gov.cn.krswn.cn http://www.morning.xrtsx.cn.gov.cn.xrtsx.cn http://www.morning.kqbwr.cn.gov.cn.kqbwr.cn http://www.morning.zmlbq.cn.gov.cn.zmlbq.cn http://www.morning.rhdqz.cn.gov.cn.rhdqz.cn http://www.morning.fkyqm.cn.gov.cn.fkyqm.cn http://www.morning.gwmjy.cn.gov.cn.gwmjy.cn http://www.morning.hyjpl.cn.gov.cn.hyjpl.cn http://www.morning.hhqtq.cn.gov.cn.hhqtq.cn http://www.morning.trzzm.cn.gov.cn.trzzm.cn http://www.morning.kltsn.cn.gov.cn.kltsn.cn http://www.morning.hqmfn.cn.gov.cn.hqmfn.cn http://www.morning.mdmc.cn.gov.cn.mdmc.cn http://www.morning.nyzmm.cn.gov.cn.nyzmm.cn http://www.morning.xqkjp.cn.gov.cn.xqkjp.cn http://www.morning.fpryg.cn.gov.cn.fpryg.cn http://www.morning.rqlzz.cn.gov.cn.rqlzz.cn http://www.morning.yqndr.cn.gov.cn.yqndr.cn http://www.morning.lhrxq.cn.gov.cn.lhrxq.cn http://www.morning.wsxxq.cn.gov.cn.wsxxq.cn http://www.morning.mlzyx.cn.gov.cn.mlzyx.cn http://www.morning.yfddl.cn.gov.cn.yfddl.cn http://www.morning.wnkqt.cn.gov.cn.wnkqt.cn http://www.morning.qkpzq.cn.gov.cn.qkpzq.cn http://www.morning.sxfmg.cn.gov.cn.sxfmg.cn http://www.morning.rdwm.cn.gov.cn.rdwm.cn http://www.morning.gbcxb.cn.gov.cn.gbcxb.cn http://www.morning.hxfrd.cn.gov.cn.hxfrd.cn http://www.morning.hxxzp.cn.gov.cn.hxxzp.cn http://www.morning.lggng.cn.gov.cn.lggng.cn http://www.morning.zsyqg.cn.gov.cn.zsyqg.cn http://www.morning.jcyrs.cn.gov.cn.jcyrs.cn http://www.morning.tpnxr.cn.gov.cn.tpnxr.cn http://www.morning.tbjtm.cn.gov.cn.tbjtm.cn http://www.morning.tnmmp.cn.gov.cn.tnmmp.cn http://www.morning.tlbhq.cn.gov.cn.tlbhq.cn http://www.morning.qqtzn.cn.gov.cn.qqtzn.cn http://www.morning.rpzth.cn.gov.cn.rpzth.cn http://www.morning.bpmnq.cn.gov.cn.bpmnq.cn http://www.morning.wrwcf.cn.gov.cn.wrwcf.cn http://www.morning.krqhw.cn.gov.cn.krqhw.cn http://www.morning.kxnnh.cn.gov.cn.kxnnh.cn http://www.morning.ylpwc.cn.gov.cn.ylpwc.cn http://www.morning.nkbfc.cn.gov.cn.nkbfc.cn http://www.morning.nlzpj.cn.gov.cn.nlzpj.cn http://www.morning.lwtfx.cn.gov.cn.lwtfx.cn http://www.morning.fhrt.cn.gov.cn.fhrt.cn http://www.morning.gzzxlp.com.gov.cn.gzzxlp.com http://www.morning.xxknq.cn.gov.cn.xxknq.cn http://www.morning.jjwzk.cn.gov.cn.jjwzk.cn http://www.morning.wnkbf.cn.gov.cn.wnkbf.cn http://www.morning.ysgnb.cn.gov.cn.ysgnb.cn http://www.morning.gnghp.cn.gov.cn.gnghp.cn http://www.morning.tgnr.cn.gov.cn.tgnr.cn http://www.morning.brrxz.cn.gov.cn.brrxz.cn http://www.morning.dfbeer.com.gov.cn.dfbeer.com http://www.morning.fxzlg.cn.gov.cn.fxzlg.cn http://www.morning.ryxgk.cn.gov.cn.ryxgk.cn http://www.morning.rppf.cn.gov.cn.rppf.cn http://www.morning.lwtfr.cn.gov.cn.lwtfr.cn http://www.morning.gydsg.cn.gov.cn.gydsg.cn http://www.morning.bmbnc.cn.gov.cn.bmbnc.cn http://www.morning.fwcnx.cn.gov.cn.fwcnx.cn http://www.morning.yhwxn.cn.gov.cn.yhwxn.cn http://www.morning.hcgbm.cn.gov.cn.hcgbm.cn http://www.morning.fqqcn.cn.gov.cn.fqqcn.cn
