当前位置：服务支持 > 软件文章 > OpenFace代码阅读笔记（2）：LandmarkDetector初始化、关键函数及论文公式与代码对应关系

OpenFace代码阅读笔记（2）：LandmarkDetector初始化、关键函数及论文公式与代码对应关系

阅读数 3

Openface代码阅读------LandmarkDetector::DetectLandmarksInVideo的算法公式和程序对应关系.

CLNF in OpenFace (upgrade version of CLM framework)

paper: CLNF

code: OpenFace

1. init detection according to arguments

step 1.

// 将 main 函数的输入参数保存到 arguments vector<string> arguments = get_arguments(argc, argv);

step 2.// 从 arguments 解析与landmark detection相关的parameters，得到 det_parameters

LandmarkDetector::FaceModelParameters det_parameters(arguments);

det_parameters.model_location = root/model_path

其中：

root 在构造函数中设置为 path(arguments[0]).parent_path()，即执行exe文件所在的目录。

model_path的初始值在 FaceModelParameters::init() 中设置为 model_location = "model/main_clnf_general.txt";

调用exe时，可以用 -mloc 选项设置 model_path。

step 3.// 根据 det_parameters 到相应地址读取 modules。

// The modules that are being used for tracking

LandmarkDetector::CLNF face_model(det_parameters.model_location);

默认情况下 det_parameters.model_location = "./model/main_clnf_general.txt"

将 main_clnf_general.txt 涉及的所有txt文件展开如下：

main_clnf_general.txt ---LandmarkDetector clnf_general.txt------PDM pdms/In-the-wild_aligned_PDM_68.txt---------# The mean values of the components (in mm) ...---------# The principal components (eigenvectors) of identity or combined identity and expression model ...---------# The variances of the components (eigenvalues) of identity or combined identity and expression model ...------Triangulations tris_68.txt---------# Number of triangulations---------# Triangulation 1---------# Triangulation 2---------# Triangulation ...---------# Triangulation 7------PatchesCCNF patch_experts/ccnf_patches_0.25_general.txt (Binary file)------PatchesCCNF patch_experts/ccnf_patches_0.35_general.txt (Binary file)------PatchesCCNF patch_experts/ccnf_patches_0.5_general.txt (Binary file)---LandmarkDetector_part model_inner/main_clnf_inner.txt inner 17 0 18 1 19 2 20 3 21 4 22 5 23 6 24 7 25 8 26 9 27 10 28 11 29 12 30 13 31 14 32 15 33 16 34 17 35 18 36 19 37 20 38 21 39 22 40 23 41 24 42 25 43 26 44 27 45 28 46 29 47 30 48 31 49 32 50 33 51 34 52 35 53 36 54 37 55 38 56 39 57 40 58 41 59 42 60 43 61 44 62 45 63 46 64 47 65 48 66 49 67 50------LandmarkDetector clnf_inner.txt---------PDM pdms/pdm_51_inner.txt---------PatchesCCNF patch_experts/ccnf_patches_1.00_inner.txt (Binary file)---LandmarkDetector_part model_eye/main_clnf_synth_left.txt left_eye_28 36 8 37 10 38 12 39 14 40 16 41 18------LandmarkDetector clnf_left_synth.txt---------PDM pdms/pdm_28_l_eye_3D_closed.txt---------PatchesCCNF patch_experts/left_ccnf_patches_1.00_synth_lid_.txt---------PatchesCCNF patch_experts/left_ccnf_patches_1.50_synth_lid_.txt---LandmarkDetector_part model_eye/main_clnf_synth_right.txt right_eye_28 42 8 43 10 44 12 45 14 46 16 47 18------LandmarkDetector clnf_right_synth.txt---------PDM pdms/pdm_28_eye_3D_closed.txt---------PatchesCCNF patch_experts/ccnf_patches_1.00_synth_lid_.txt---------PatchesCCNF patch_experts/ccnf_patches_1.50_synth_lid_.txt---FaceDetConversion haarAlign.txt------3------4------6------1.000000 0.000000 0.000000 0.000000 ------0.000000 1.000000 0.000000 0.000000 ------0.055858 0.229781 0.871248 0.892778 ---DetectionValidator detection_validation/validator_general_68.txt (Binary file)

2. call stack

detection_success = LandmarkDetector::DetectLandmarksInVideo(grayscale_image, face_model, det_parameters);

>	FeatureExtraction.exe!LandmarkDetector::CLNF::Fit(const cv::Mat_<unsigned char> & im, const cv::Mat_<float> & depthImg, const std::vector<int,std::allocator<int> > & window_sizes, const LandmarkDetector::FaceModelParameters & parameters) Line 759	C++ 	FeatureExtraction.exe!LandmarkDetector::CLNF::DetectLandmarks(const cv::Mat_<unsigned char> & image, const cv::Mat_<float> & depth, LandmarkDetector::FaceModelParameters & params) Line 571	C++ 	FeatureExtraction.exe!LandmarkDetector::DetectLandmarksInVideo(const cv::Mat_<unsigned char> & grayscale_image, const cv::Mat_<float> & depth_image, LandmarkDetector::CLNF & clnf_model, LandmarkDetector::FaceModelParameters & params) Line 362	C++ 	FeatureExtraction.exe!LandmarkDetector::DetectLandmarksInVideo(const cv::Mat_<unsigned char> & grayscale_image, LandmarkDetector::CLNF & clnf_model, LandmarkDetector::FaceModelParameters & params) Line 424	C++ 	FeatureExtraction.exe!main(int argc, char * * argv) Line 601	C++

3. Constrained Local Model(CLM) Face Landmark Detection

Our approach uses the Constrained Local Model (CLM) framework.

There are three main parts to a CLM:

Part A: a point distribution model (PDM),

which is parameter p in formula (1).

calc PDM according to ground true landmarks in train dataset.

OpenFace read PDM information from txt files.

本例中从如下文件中读取 PDM。

./model/pdms/In-the-wild_aligned_PDM_68.txt

Part B: patch experts,

which is used to calculate xi in formula (1).

Refer to section 3.1. Local Neural Field patch expert.

LandmarkDetectorModel.cpp line:722

		// The patch expert response computation
		if(scale != window_sizes.size() - 1)
		{
			patch_experts.Response(patch_expert_responses, sim_ref_to_img, sim_img_to_ref, im, depth_img_no_background, pdm, params_global, params_local, window_size, scale);
		}

Part C: the fitting approach,

which is used to find optimized p* in formula (1).

Refer to section 3.2. Non-uniform RLMS.

LandmarkDetectorModel.cpp line:759  // the actual optimisation step		this->NU_RLMS(params_global, params_local, patch_expert_responses, cv::Vec6d(params_global), params_local.clone(), current_shape, sim_img_to_ref, sim_ref_to_img, window_size, view_id, true, scale, this->landmark_likelihoods, tmp_parameters);

免责声明：本文系网络转载或改编，未找到原创作者，版权归原作者所有。如涉及版权，请联系删

返回上级列表