SLAM Hive-Evaluate

Wiki Link

Total mapping tasks number: 6

Contains (mapping task id, evaluation id): [ (1341, 985), (1342, 986), (1343, 987), (1345, 989), (1346, 994), (1347, 995) ]

Current mappingtask id: 1341

lio-sam with kitti_2011_09_30_drive_0028_synced_full

Algorithm parameters
Parameter name	Parameter value
edgeThreshold	1.0
surfThreshold	0.1
edgeFeatureMinValidNum	10
surfFeatureMinValidNum	100
odometrySurfLeafSize	0.4
mappingCornerLeafSize	0.2
mappingSurfLeafSize	0.4
z_tollerance	1000
rotation_tollerance	1000
numberOfCores	4
mappingProcessInterval	0.15
surroundingkeyframeAddingDistThreshold	1.0
surroundingkeyframeAddingAngleThreshold	0.2
surroundingKeyframeDensity	2.0
surroundingKeyframeSearchRadius	50.0
loopClosureFrequency	1.0
surroundingKeyframeSize	50
historyKeyframeSearchRadius	15.0
historyKeyframeSearchTimeDiff	30.0
historyKeyframeSearchNum	25
historyKeyframeFitnessScore	0.3
gpsCovThreshold	2.0
poseCovThreshold	25.0
downsampleRate	2
pointCloudTopic	points_raw
imuTopic	imu_raw
savePCD	False
savePCDDirectory	/Downloads/LOAM/
loopClosureEnableFlag	True
navsatfrequency	50
navsatwait_for_datum	False
navsatdelay	0.0
navsatmagnetic_declination_radians	0
navsatyaw_offset	0
navsatzero_altitude	True
navsatbroadcast_utm_transform	False
navsatbroadcast_utm_transform_as_parent_frame	False
navsatpublish_filtered_gps	False
ekf_gpspublish_tf	False
ekf_gpsfrequency	50
ekf_gpstwo_d_mode	False
ekf_gpssensor_timeout	0.01
GpsElevation	False
gpsTopic	odometry/gpsz

Dataset parameters
Parameter name	Parameter value
imuGyrNoise	0.0015636343949698187
imuAccNoise	0.003993957088823881
imuGyrBiasN	3.564031869636761e-05
imuAccBiasN	6.435665935353257e-05
imuGravity	9.80511
sensor	velodyne
N_SCAN	64
Horizon_SCAN	1800
lidarMinRange	1.0
lidarMaxRange	1000.0
imuRPYWeight	0.01
extrinsicTrans	[-0.8086759, 0.3195559, -0.7997231]
extrinsicRPY	[0.9999976, 0.0007553071, -0.002035826, -0.0007854027, 0.9998898, -0.01482298, 0.002024406, 0.01482454, 0.9998881]
extrinsicRot	[0.9999976, 0.0007553071, -0.002035826, -0.0007854027, 0.9998898, -0.01482298, 0.002024406, 0.01482454, 0.9998881]
lidar_frequency	10
imu_frequency	100

Algorithm remaps
Parameter name	Parameter value
None	None

Dataset remaps
Parameter name	Parameter value
None	None

ATE and RPE Error Statistics
	RMSE	Mean	Median	Std	Min	Max	SSE
ATE	6.24148	5.65454	5.38875	2.64238	0.799605	15.2009	76236.9
RPE	1.45236	1.43469	1.44984	0.225818	0.912529	2.41515	432.414

Computing resource usage
Max CPU (core)	Mean CPU (core)	Max Memory (MB)
4.50759	2.32295	1330.19

Current mappingtask id: 1342

a-loam with kitti_2011_09_30_drive_0028_synced_full

Algorithm parameters
Parameter name	Parameter value
mapping_skip_frame	1
minimum_range	5
mapping_line_resolution	0.4
mapping_plane_resolution	0.8
aloam_topic	/points_raw
scan_line	64

Dataset parameters
Parameter name	Parameter value
lidar_frequency	10

Algorithm remaps
Parameter name	Parameter value
None	None

Dataset remaps
Parameter name	Parameter value
None	None

ATE and RPE Error Statistics
	RMSE	Mean	Median	Std	Min	Max	SSE
ATE	9.06253	7.69879	6.05556	4.781	0.792863	25.4023	329996.0
RPE	1.41461	1.40565	1.40385	0.158935	0.779026	2.78386	3373.88

Computing resource usage
Max CPU (core)	Mean CPU (core)	Max Memory (MB)
3.36908	2.84443	953.578

Current mappingtask id: 1343

loam with kitti_2011_09_30_drive_0028_synced_full

Algorithm parameters
Parameter name	Parameter value
scanPeriod	0.1

Dataset parameters
Parameter name	Parameter value
lidar	HDL-64E
lidar_frequency	10

Algorithm remaps
Parameter name	Parameter value
None	None

Dataset remaps
Parameter name	Parameter value
None	None

ATE and RPE Error Statistics
	RMSE	Mean	Median	Std	Min	Max	SSE
ATE	8.76339	8.36998	8.39912	2.59624	1.43018	14.9475	113890.0
RPE	1.52679	1.46266	1.39094	0.437869	0.715747	2.74783	237.772

Computing resource usage
Max CPU (core)	Mean CPU (core)	Max Memory (MB)
2.90709	2.22867	4691.79

Current mappingtask id: 1345

lego-loam-imu with kitti_2011_09_30_drive_0028_synced_full

Algorithm parameters
Parameter name	Parameter value
pointcloud_topic	/points_raw
imu_topic	/imu_raw

Dataset parameters
Parameter name	Parameter value
lidar_frequency	10
imu_frequency	100

Algorithm remaps
Parameter name	Parameter value
None	None

Dataset remaps
Parameter name	Parameter value
None	None

ATE and RPE Error Statistics
	RMSE	Mean	Median	Std	Min	Max	SSE
ATE	9.41201	7.80973	6.13596	5.25299	0.983575	28.6638	355938.0
RPE	0.304684	0.181446	0.0756975	0.244765	0.0117661	2.06005	154.937

Computing resource usage
Max CPU (core)	Mean CPU (core)	Max Memory (MB)
2.4662	1.43371	1066.25

Current mappingtask id: 1346

ndt-loam with kitti_2011_09_30_drive_0028_synced_full

Algorithm parameters
Parameter name	Parameter value
distance_near_thresh	0.5
distance_far_thresh	100.0
downsample_resolution	0.1
statistical_mean_k	30
statistical_stddev	1.2
radius_radius	0.5
radius_min_neighbors	5
keyframe_delta_trans	10.0
keyframe_delta_angle	0.17
keyframe_delta_time	1.0
scan_line	64
mapping_skip_frame	1
minimum_range	5
mapping_line_resolution	0.4
mapping_plane_resolution	0.8
seq	4
g2o_solver_num_iterations	1024
distance_thresh	20.0
accum_distance_thresh	100.0
min_edge_interval	50.0
fitness_score_thresh	2.0
ndt_resolution	1.0
ndt_num_threads	8
gps_edge_robust_kernel_size	1.0
gps_edge_stddev_xy	20.0
gps_edge_stddev_z	5.0
imu_orientation_edge_stddev	1.0
imu_acceleration_edge_stddev	1.0
floor_edge_stddev	100.0
odometry_edge_robust_kernel_size	1.0
loop_closure_edge_robust_kernel_size	1.0
const_stddev_x	0.5
const_stddev_q	0.1
var_gain_a	20.0
min_stddev_x	0.1
max_stddev_x	5.0
min_stddev_q	0.05
max_stddev_q	0.2
graph_update_interval	10.0
max_keyframes_per_update	20
map_cloud_update_interval	10.0
map_cloud_resolution	0.5
pointcloud_topic	/points_raw
enable_floor_detection	False
enable_gps	False
enable_imu_acc	False
enable_imu_ori	False
base_link_frame	base_link
use_distance_filter	True
downsample_method	VOXELGRID
outlier_removal_method	RADIUS
use_angle_calibration	False
odom_frame_id	odom
calib_file	home/chenshoubin/slam_ws/src/lv_slam/config/kitti_calib/calib04-12.txt
lfa_output_path	/home/chenshoubin/slam_ws/src/lv_slam/data/kitti_lv_dlo_lfa_ggo
g2o_solver_type	lm_var_cholmod
registration_method	NDT_OMP
ndt_nn_search_method	DIRECT7
gps_edge_robust_kernel	Huber
imu_orientation_edge_robust_kernel	Huber
imu_acceleration_edge_robust_kernel	Huber
floor_edge_robust_kernel	Huber
odometry_edge_robust_kernel	Huber
loop_closure_edge_robust_kernel	Huber
use_const_inf_matrix	True

Dataset parameters
Parameter name	Parameter value
lidar_frequency	10

Algorithm remaps
Parameter name	Parameter value
None	None

Dataset remaps
Parameter name	Parameter value
None	None

ATE and RPE Error Statistics
	RMSE	Mean	Median	Std	Min	Max	SSE
ATE	15.2738	11.776	7.95584	9.72692	1.67536	49.3917	937350.0
RPE	1.42332	1.41422	1.41212	0.160652	0.801463	2.72385	3395.29

Computing resource usage
Max CPU (core)	Mean CPU (core)	Max Memory (MB)
5.27625	4.17716	5214.83

Current mappingtask id: 1347

ndt-loam with kitti_2011_09_30_drive_0028_synced_full

Algorithm parameters
Parameter name	Parameter value
distance_near_thresh	0.5
distance_far_thresh	100.0
downsample_resolution	0.1
statistical_mean_k	30
statistical_stddev	1.2
radius_radius	0.5
radius_min_neighbors	5
keyframe_delta_trans	10.0
keyframe_delta_angle	0.17
keyframe_delta_time	1.0
scan_line	64
mapping_skip_frame	1
minimum_range	5
mapping_line_resolution	0.4
mapping_plane_resolution	0.8
seq	4
g2o_solver_num_iterations	1024
distance_thresh	20.0
accum_distance_thresh	100.0
min_edge_interval	50.0
fitness_score_thresh	2.0
ndt_resolution	1.0
ndt_num_threads	8
gps_edge_robust_kernel_size	1.0
gps_edge_stddev_xy	20.0
gps_edge_stddev_z	5.0
imu_orientation_edge_stddev	1.0
imu_acceleration_edge_stddev	1.0
floor_edge_stddev	100.0
odometry_edge_robust_kernel_size	1.0
loop_closure_edge_robust_kernel_size	1.0
const_stddev_x	0.5
const_stddev_q	0.1
var_gain_a	20.0
min_stddev_x	0.1
max_stddev_x	5.0
min_stddev_q	0.05
max_stddev_q	0.2
graph_update_interval	10.0
max_keyframes_per_update	20
map_cloud_update_interval	10.0
map_cloud_resolution	0.5
pointcloud_topic	/points_raw
enable_floor_detection	False
enable_gps	False
enable_imu_acc	False
enable_imu_ori	False
base_link_frame	base_link
use_distance_filter	True
downsample_method	VOXELGRID
outlier_removal_method	RADIUS
use_angle_calibration	False
odom_frame_id	odom
calib_file	home/chenshoubin/slam_ws/src/lv_slam/config/kitti_calib/calib04-12.txt
lfa_output_path	/home/chenshoubin/slam_ws/src/lv_slam/data/kitti_lv_dlo_lfa_ggo
g2o_solver_type	lm_var_cholmod
registration_method	NDT_OMP
ndt_nn_search_method	DIRECT7
gps_edge_robust_kernel	Huber
imu_orientation_edge_robust_kernel	Huber
imu_acceleration_edge_robust_kernel	Huber
floor_edge_robust_kernel	Huber
odometry_edge_robust_kernel	Huber
loop_closure_edge_robust_kernel	Huber
use_const_inf_matrix	True
imu_topic	/imu_raw

Dataset parameters
Parameter name	Parameter value
lidar_frequency	10
imu_frequency	100

Algorithm remaps
Parameter name	Parameter value
None	None

Dataset remaps
Parameter name	Parameter value
None	None

ATE and RPE Error Statistics
	RMSE	Mean	Median	Std	Min	Max	SSE
ATE	15.2738	11.776	7.95584	9.72692	1.67536	49.3917	937350.0
RPE	1.42332	1.41422	1.41212	0.160652	0.801463	2.72385	3395.29

Computing resource usage
Max CPU (core)	Mean CPU (core)	Max Memory (MB)
4.71194	3.7774	5559.0

SLAM Hive: A SLAM Evaluation Benchmark System

Total mapping tasks number: 6

Contains (mapping task id, evaluation id): [ (1341, 985), (1342, 986), (1343, 987), (1345, 989), (1346, 994), (1347, 995) ]

Current mappingtask id: 1341

lio-sam with kitti_2011_09_30_drive_0028_synced_full

Current mappingtask id: 1342

a-loam with kitti_2011_09_30_drive_0028_synced_full

Current mappingtask id: 1343

loam with kitti_2011_09_30_drive_0028_synced_full

Current mappingtask id: 1345

lego-loam-imu with kitti_2011_09_30_drive_0028_synced_full

Current mappingtask id: 1346

ndt-loam with kitti_2011_09_30_drive_0028_synced_full

Current mappingtask id: 1347

ndt-loam with kitti_2011_09_30_drive_0028_synced_full