Kinect One和Matlab - 数据流显示

Question

当前版本的Matlab本身不支持Kinect One。在Stackoverflow上有几篇帖子询问如何在Matlab环境中渲染特定的数据流（深度，骨架或rgb）。有没有人有任何建议或评论如何记录/显示Matlab内的数据？

Answer 1

本文旨在解释如何在Matlab中渲染Kinect One的数据流。我一直倾向于单独记录数据，而不是使用Matlab。对于那些正在挣扎的人来说，这是一个小小的帮助指南。 Kinect One不兼容Matlab（当前）。在这里，我发布示例如何在Matlab中渲染骨架，RGB和深度图像。这个想法是允许你用Matlab在数据中执行操作。请注意，我只发布了功能，您需要生成您自己的显示屏。此外，您必须分别记录数据（Kinect以外）必须。这些代码可能需要更改最新的SDK版本。你需要自己完成一些工作，但这些代码应该是一个很好的指标。

请注意，在我将文件更新到最新的SDK后的几周内，我将发布一个完整的界面（如下所示）。

这个职位将通过以下方式进行格式化。对于每个流，我将在Matlab中发布函数进行渲染，以及如何从Kinect捕获数据的示例.cpp。代码有点混乱，因为我已经从现有的项目中提取它。

如果您有任何问题或需要改进的意见，请发布。

骨架：

.CPP片段

private void saveSkel(long timeStamp, Body body) 
     { 

      string filePath = yourPath + '\\' + timeStamp + ".txt"; 

      StreamWriter cooStream = new StreamWriter(filePath, false); 

      IReadOnlyDictionary<JointType, Joint> joints = body.Joints; 

      Dictionary<JointType, Point> jointPoints = new Dictionary<JointType, Point>();  

      foreach (JointType jointType in joints.Keys) 
      { 
       //Camera space points 
       ColorSpacePoint depthSpacePoint = this.coordinateMapper.MapCameraPointToColorSpace(joints[jointType].Position); 

       cooStream.WriteLine(joints[jointType].JointType +" " + joints[jointType].TrackingState + " " + joints[jointType].Position.X + " " + joints[jointType].Position.Y + " " + joints[jointType].Position.Z + " " + depthSpacePoint.X + " " + depthSpacePoint.Y); 
      } 
       //If we want to record both hand states 
       if (handLassO == true) 
       { 
        string wrtLineData = "LeftHand " + body.HandLeftState + " RightHand " + body.HandRightState; 
        cooStream.WriteLine(wrtLineData); 

       } 

      cooStream.Close(); 
     } 

Matlab代码

功能1：

function handle = displayKinectSkeleton(ax, Skels) 
%This function displays the Kinect skeleton 
% 
%Input: 
% ax - axis of the skeleton 
% Skels - skeleton file [format: 3*25 matrix (X;Y;Z)] 
% 
%Output: 
% handle - return the kinect handle 

%Index the joints 
joints.SpineBase = 1; 
joints.SpineMid =2; 
joints.Neck =3; 
joints.Head =4; 
joints.ShoulderLeft =5; 
joints.ElbowLeft =6; 
joints.WristLeft =7; 
joints.HandLeft =8; 
joints.ShoulderRight =9; 
joints.ElbowRight =10; 
joints.WristRight =11; 
joints.HandRight =12; 
joints.HipLeft =13; 
joints.KneeLeft =14; 
joints.AnkleLeft =15; 
joints.FootLeft =16; 
joints.HipRight =17; 
joints.KneeRight =18; 
joints.AnkleRight =19; 
joints.FootRight =20; 
joints.SpineShoulder =21; 
joints.HandTipLeft =22; 
joints.ThumbLeft =23; 
joints.HandTipRight =24; 
joints.ThumbRight =25; 
joints.position_count = 25; 

%Generate connections 
skeleton_conn = [ ... 
    %Torso 
    joints.SpineBase, joints.SpineMid; ... %1 
    joints.SpineMid,joints.SpineShoulder;... %2 
    joints.SpineShoulder, joints.Neck;... %3 
    joints.Neck,joints.Head;... %4 

    %Left arm 
    joints.SpineShoulder,joints.ShoulderLeft;... %5 
    joints.ShoulderLeft, joints.ElbowLeft;... %6 
    joints.ElbowLeft, joints.WristLeft;... %7 
    joints.WristLeft, joints.HandLeft;... %8 

    %Right arm 
    joints.SpineShoulder, joints.ShoulderRight;... %9 
    joints.ShoulderRight, joints.ElbowRight;...%10 
    joints.ElbowRight, joints.WristRight;... %11 
    joints.WristRight,joints.HandRight;... %12 

    %Left leg 
    joints.SpineBase, joints.HipLeft;... %13 
    joints.HipLeft, joints.KneeLeft;... %14 
    joints.KneeLeft,joints.AnkleLeft;... %15 
    joints.AnkleLeft,joints.FootLeft;... %16 

    %Right legs 
    joints.SpineBase, joints.HipRight;... %17 
    joints.HipRight, joints.KneeRight;... %18 
    joints.KneeRight, joints.AnkleRight;... %19 
    joints.AnkleRight, joints.FootRight; %20 

    %Wrist 
    joints.HandLeft, joints.HandTipLeft;... %21 
    joints.WristLeft, joints.ThumbLeft;... %22 

    %wrist 
    joints.HandRight, joints.HandTipRight;... %23 
    joints.WristRight, joints.ThumbRight;... %24 

    ]; 

%Orignal XYZ points.I'be reshaped as I use the format elsewhere. 
xyz_shape = reshape(Skels, 3, 25)'; %I reshape as I use it for some different) 
x = xyz_shape(:,1); 
y = xyz_shape(:,2); 
z = xyz_shape(:,3); 

%Join the skeleton 
xyz_step=[x y z]; 

%Plot the skeleton 
handle = scatter3(ax, xyz_step(:,1), xyz_step(:,2), xyz_step(:,3), 'bo'); 

%Generate the limb connections 
%Torso 
torsoColor = 'r'; 
handle = displayLimbs(ax, xyz_step', skeleton_conn(1:4,:), torsoColor, 2.5); 

%Left Arm 
leftArmColor = 'g'; 
handle = displayLimbs(ax, xyz_step', skeleton_conn(5:7,:), leftArmColor, 2.5); 
handle = displayLimbs(ax, xyz_step', skeleton_conn(8:8,:), leftArmColor, 2.5); 

%Right Arm 
rightArmColor = 'b'; 
handle = displayLimbs(ax, xyz_step', skeleton_conn(9:11,:), rightArmColor, 2.5); 
handle = displayLimbs(ax, xyz_step', skeleton_conn(12:12,:), rightArmColor, 2.5); 

%Legs 
leftLegColor = 'y'; 
rightLegColor = 'm'; 
handle = displayLimbs(ax, xyz_step', skeleton_conn(13:14,:), leftLegColor, 2.5); 
handle = displayLimbs(ax, xyz_step', skeleton_conn(14:16,:), leftLegColor, 2.5); 

handle = displayLimbs(ax, xyz_step', skeleton_conn(17:18,:), rightLegColor, 2.5); 
handle = displayLimbs(ax, xyz_step', skeleton_conn(18:20,:), rightLegColor, 2.5); 

%wrist 
wristColor = 'r'; 
handle = displayLimbs(ax, xyz_step', skeleton_conn(21:21,:), wristColor, 2.5); 
handle = displayLimbs(ax, xyz_step', skeleton_conn(22:22,:), wristColor, 2.5); 
handle = displayLimbs(ax, xyz_step', skeleton_conn(23:23,:), wristColor, 2.5); 
handle = displayLimbs(ax, xyz_step', skeleton_conn(24:24,:), wristColor, 2.5); 
end 

功能2：

function [ handle ] = displayLimbs(ax, xyz, jointLoc, color, width) 
%This function joins specific joints together with the line function. 
% 
%Input: 
% ax - axis location 
% xyz - the xyz location for the time instance 
% jointLoc - joint index locations 
% color - color of the lines 
% width - width of the lines 
% 
%Output: 
% handle - Kinect handle 
handle = line(xyz(1,jointLoc), xyz(2,jointLoc), xyz(3,jointLoc), 'Color', color, 'Parent', ax, 'LineWidth', width); 
end 

深度：

.CPP片段 *这是如何保存深度数据的示例。

//Save the depth data 

    string filePath = yourPath + '\\' + "depth" + frame.RelativeTime.ToString() + ".bin"; 

    using (FileStream streamDepth = new FileStream(filePath, FileMode.Create)) 
     { 
     using (BinaryWriter depthWriter = new BinaryWriter(streamDepth)) 
     { 
     depthWriter.Write(this.pixelsDepth); 
     depthWriter.Close(); 
     } 
     } 

Matlab代码

功能1：
*使用上述的.cpp代码。阅读并显示如下。

function [depthImg] = ReadKiectDepthIndivi(loc) 
%This function reads in a Kinect depth image from a bin file 
% 
%Input: 
% loc - location of the bin file 
%Output: 
% depthImg - depth image 

%Read in the file 
fid = fopen(loc); %read the file in 
[A, count] = fread(fid, 'uint16=>uint16',2); %load and shift 
fclose(fid); %close the stream 

%Convert into structure 
dims = [512 424]; %dims of the depth 

depth = double(A); %convert to double for imaging 

[depthImg.X, depthImg.Y] = meshgrid([1:dims(1)], [1:dims(2)]); %obtain x and y 
depthImg.Z = reshape(depth, dims(1), dims(2))'; %generate depth 
%temp.Z(temp.Z==0)=NaN; % noise clean up (if required) 

end 

渲染并使用下面的命令显示：

surf(displayHandles.depthHandle, depthImg, depthImg, depthImg, 'EdgeColor','None', 'FaceAlpha', 0.5); 

RGB：

.CPP片段 *对于速度和效率保存为一个bin文件。但是，取决于你的机器。您可能需要降低采样率。

string filePath = yourPath + '\\' + "image" + frame.RelativeTime.ToString() + ".bin"; 

using (FileStream streamRGB = new FileStream(filePath, FileMode.Create)) 
{ 
    using (BinaryWriter rgbWriter = new BinaryWriter(streamRGB)) 
    { 
    rgbWriter.Write(this.pixels); 
    rgbWriter.Close(); 
} 

Matlab代码

功能1：

function [rgbImg] = ReadKinectRGBIndvid(loc) 
%This function read's in a Kinect RGB image from a bin file 
% 
%Input: 
% loc - location of the bin file 
%Ouput: 
% depthImg - depth image 
%History: 
% Created by Dan 11/03/14 (based on JD version) 

%Read in the image file 
fid = fopen(loc); 
[A, count] = fread(fid, 'uint8=>uint8'); 
fclose(fid); 

dims = [1920 1080]; 
rgbImg(:,:,3) = reshape(A(1:4:end), dims(1), dims(2)); 
rgbImg(:,:,2) = reshape(A(2:4:end), dims(1), dims(2)); 
rgbImg(:,:,1) = reshape(A(3:4:end), dims(1), dims(2)); 

imageRGB = permute(rgbImg, [2 1 3]); 

end 

渲染和显示使用下面的命令（或某种形式的）：

RGB = imrotate(RGB ,-90); 
RGB = flipdim(RGB,2); 
image(RGB, 'Parent',displayHandles.rgbHandle);