做网站的流程分析,软件开发行业市场分析,新乡发布最新通告,共享看世界新域名一、DataNode中的数据情况
数据位置
/opt/module/hadoop-3.1.3/data/dfs/data/current/BP-823420375-192.168.31.102-1714395693863/current/finalized/subdir0/subdir0块信息 每个块信息#xff0c;由两个文件保存#xff0c;xxx.meta保存的是数据长度、校验和、时间戳由两个文件保存xxx.meta保存的是数据长度、校验和、时间戳另外一个是保存真实数据内容的。
二、工作流程
DataNode和NameNode的工作流程比较简单主要内容是DN向NN注册DN定时向NN汇报状态
1、周期上报的相关默认配置
hdfs-default.xml 上报NN时间配置
propertynamedfs.blockreport.intervalMsec/namevalue21600000/valuedescriptionDetermines block reporting interval in milliseconds./description
/propertyDN自查时间配置
propertynamedfs.datanode.directoryscan.interval/namevalue21600s/valuedescriptionInterval in seconds for Datanode to scan data directories andreconcile the difference between blocks in memory and on the disk.Support multiple time unit suffix(case insensitive), as describedin dfs.heartbeat.interval./description
/property一般这两个配置时间是一样的工作流程是DN先自查然后将结果汇报给NN
2、超时时长相关默认配置
计算公式 心跳时间配置
propertynamedfs.heartbeat.interval/namevalue3s/valuedescriptionDetermines datanode heartbeat interval in seconds.Can use the following suffix (case insensitive):ms(millis), s(sec), m(min), h(hour), d(day)to specify the time (such as 2s, 2m, 1h, etc.).Or provide complete number in seconds (such as 30 for 30 seconds)./description
/property心跳检测时间参数
propertynamedfs.namenode.heartbeat.recheck-interval/namevalue300000/valuedescriptionThis time decides the interval to check for expired datanodes.With this value and dfs.heartbeat.interval, the interval ofdeciding the datanode is stale or not is also calculated.The unit of this configuration is millisecond./description
/property
