基于颜色和纹理的阴影检测

Question

我正在做一个关于视频监控中的跟踪和对象分类的项目，并且我从移除前景对象的阴影方面遇到了一些困难。 问题是我不能只使用我想要的任何方法，我必须使用本文中描述的相同方法Shadow removal with blob based morphological reconstruction。

在文章中他们使用混合阴影去除方法 - 基于RGB颜色的检测和基于纹理的检测。基于颜色和纹理的过程是并行使用的，然后是结合两者结果的断言过程。

所以我想得到一些帮助与阴影消除Matlab代码。

到目前为止我的代码

function Tracking_Objects() 

% Create System objects used for reading video, detecting moving objects, 
% and displaying the results. 
obj = setupSystemObjects(); 

tracks = initializeTracks(); % Create an empty array of tracks. 

nextId = 1; % ID of the next track 

% Detect moving objects, and track them across video frames. 
while ~isDone(obj.reader) 
    frame = readFrame(); 
    [centroids, bboxes, mask] = detectObjects(frame); 
    predictNewLocationsOfTracks(); 
    [assignments, unassignedTracks, unassignedDetections] = ... 
     detectionToTrackAssignment(); 

    updateAssignedTracks(); 
    updateUnassignedTracks(); 
    deleteLostTracks(); 
    createNewTracks(); 

    displayTrackingResults(); 
end 


%% Create System Objects 

    function obj = setupSystemObjects() 
     % Initialize Video I/O 
     % Create objects for reading a video from a file, drawing the tracked 
     % objects in each frame, and playing the video. 

     % Create a video file reader. 
     obj.reader = vision.VideoFileReader('input.avi'); 

     % Create two video players, one to display the video, 
     % and one to display the foreground mask. 
     obj.videoPlayer = vision.VideoPlayer('Position', [10, 250, 700, 400]); 
     obj.maskPlayer = vision.VideoPlayer('Position', [720, 250, 700, 400]); 


     obj.detector = vision.ForegroundDetector('NumGaussians', 3, ... 
      'NumTrainingFrames', 40, 'MinimumBackgroundRatio', 0.7); 


     obj.blobAnalyser = vision.BlobAnalysis('BoundingBoxOutputPort', true, ... 
      'AreaOutputPort', true, 'CentroidOutputPort', true, ... 
      'MinimumBlobArea', 75); 
    end 

%% Initialize Tracks 

    function tracks = initializeTracks() 
     % create an empty array of tracks 
     tracks = struct(... 
      'id', {}, ... 
      'bbox', {}, ... 
      'kalmanFilter', {}, ... 
      'age', {}, ... 
      'totalVisibleCount', {}, ... 
      'consecutiveInvisibleCount', {}); 
    end 

%% Read a Video Frame 
% Read the next video frame from the video file. 
    function frame = readFrame() 
     frame = obj.reader.step(); 
    end 

%% Detect Objects 

    function [centroids, bboxes, mask] = detectObjects(frame) 

     % Detect foreground. 
     mask = obj.detector.step(frame); 

     % Apply morphological operations to remove noise and fill in holes. 
     mask = imopen(mask, strel('rectangle', [3,3])); 
     mask = imclose(mask, strel('rectangle', [15, 15])); 
     mask = imfill(mask, 'holes'); 

     % Perform blob analysis to find connected components. 
     [~, centroids, bboxes] = obj.blobAnalyser.step(mask); 
    end 

%% Predict New Locations of Existing Tracks 

    function predictNewLocationsOfTracks() 
     for i = 1:length(tracks) 
      bbox = tracks(i).bbox; 

      % Predict the current location of the track. 
      predictedCentroid = predict(tracks(i).kalmanFilter); 

      % Shift the bounding box so that its center is at 
      % the predicted location. 
      predictedCentroid = int32(predictedCentroid) - bbox(3:4)/2; 
      tracks(i).bbox = [predictedCentroid, bbox(3:4)]; 
     end 
    end 

%% Assign Detections to Tracks 

    function [assignments, unassignedTracks, unassignedDetections] = ... 
      detectionToTrackAssignment() 

     nTracks = length(tracks); 
     nDetections = size(centroids, 1); 

     % Compute the cost of assigning each detection to each track. 
     cost = zeros(nTracks, nDetections); 
     for i = 1:nTracks 
      cost(i, :) = distance(tracks(i).kalmanFilter, centroids); 
     end 

     % Solve the assignment problem. 
     costOfNonAssignment = 20; 
     [assignments, unassignedTracks, unassignedDetections] = ... 
      assignDetectionsToTracks(cost, costOfNonAssignment); 
    end 

%% Update Assigned Tracks 

    function updateAssignedTracks() 
     numAssignedTracks = size(assignments, 1); 
     for i = 1:numAssignedTracks 
      trackIdx = assignments(i, 1); 
      detectionIdx = assignments(i, 2); 
      centroid = centroids(detectionIdx, :); 
      bbox = bboxes(detectionIdx, :); 

      % Correct the estimate of the object's location 
      % using the new detection. 
      correct(tracks(trackIdx).kalmanFilter, centroid); 

      % Replace predicted bounding box with detected 
      % bounding box. 
      tracks(trackIdx).bbox = bbox; 

      % Update track's age. 
      tracks(trackIdx).age = tracks(trackIdx).age + 1; 

      % Update visibility. 
      tracks(trackIdx).totalVisibleCount = ... 
       tracks(trackIdx).totalVisibleCount + 1; 
      tracks(trackIdx).consecutiveInvisibleCount = 0; 
     end 
    end 

%% Update Unassigned Tracks 
% Mark each unassigned track as invisible, and increase its age by 1. 

    function updateUnassignedTracks() 
     for i = 1:length(unassignedTracks) 
      ind = unassignedTracks(i); 
      tracks(ind).age = tracks(ind).age + 1; 
      tracks(ind).consecutiveInvisibleCount = ... 
       tracks(ind).consecutiveInvisibleCount + 1; 
     end 
    end 

%% Delete Lost Tracks 

    function deleteLostTracks() 
     if isempty(tracks) 
      return; 
     end 

     invisibleForTooLong = 20; 
     ageThreshold = 8; 

     % Compute the fraction of the track's age for which it was visible. 
     ages = [tracks(:).age]; 
     totalVisibleCounts = [tracks(:).totalVisibleCount]; 
     visibility = totalVisibleCounts ./ ages; 

     % Find the indices of 'lost' tracks. 
     lostInds = (ages < ageThreshold & visibility < 0.6) | ... 
      [tracks(:).consecutiveInvisibleCount] >= invisibleForTooLong; 

     % Delete lost tracks. 
     tracks = tracks(~lostInds); 
    end 

%% Create New Tracks 

    function createNewTracks() 
     centroids = centroids(unassignedDetections, :); 
     bboxes = bboxes(unassignedDetections, :); 

     for i = 1:size(centroids, 1) 

      centroid = centroids(i,:); 
      bbox = bboxes(i, :); 

      % Create a Kalman filter object. 
      kalmanFilter = configureKalmanFilter('ConstantVelocity', ... 
       centroid, [200, 50], [100, 25], 100); 

      % Create a new track. 
      newTrack = struct(... 
       'id', nextId, ... 
       'bbox', bbox, ... 
       'kalmanFilter', kalmanFilter, ... 
       'age', 1, ... 
       'totalVisibleCount', 1, ... 
       'consecutiveInvisibleCount', 0); 

      % Add it to the array of tracks. 
      tracks(end + 1) = newTrack; 

      % Increment the next id. 
      nextId = nextId + 1; 
     end 
    end 

%% Display Tracking Results 
% The |displayTrackingResults| function draws a bounding box and label ID 
% for each track on the video frame and the foreground mask. It then 
% displays the frame and the mask in their respective video players. 

    function displayTrackingResults() 
     % Convert the frame and the mask to uint8 RGB. 
     frame = im2uint8(frame); 
     mask = uint8(repmat(mask, [1, 1, 3])) .* 255; 

     minVisibleCount = 8; 
     if ~isempty(tracks) 

      % Noisy detections tend to result in short-lived tracks. 
      % Only display tracks that have been visible for more than 
      % a minimum number of frames. 
      reliableTrackInds = ... 
       [tracks(:).totalVisibleCount] > minVisibleCount; 
      reliableTracks = tracks(reliableTrackInds); 

      % Display the objects. If an object has not been detected 
      % in this frame, display its predicted bounding box. 
      if ~isempty(reliableTracks) 
       % Get bounding boxes. 
       bboxes = cat(1, reliableTracks.bbox); 

       % Get ids. 
       ids = int32([reliableTracks(:).id]); 

       % Create labels for objects indicating the ones for 
       % which we display the predicted rather than the actual 
       % location. 
       labels = cellstr(int2str(ids')); 
       predictedTrackInds = ... 
        [reliableTracks(:).consecutiveInvisibleCount] > 0; 
       isPredicted = cell(size(labels)); 
       isPredicted(predictedTrackInds) = {' predicted'}; 
       labels = strcat(labels, isPredicted); 

       % Draw the objects on the frame. 
       frame = insertObjectAnnotation(frame, 'rectangle', ... 
        bboxes, labels); 

       % Draw the objects on the mask. 
       mask = insertObjectAnnotation(mask, 'rectangle', ... 
        bboxes, labels); 
      end 
     end 

     % Display the mask and the frame. 
     obj.maskPlayer.step(mask);   
     obj.videoPlayer.step(frame); 
    end 

end 

Answer 1

看来你需要修改detectObjects函数的代码来消除阴影。目前，它所做的只是从背景减除（vision.ForegroundDetector）中获取前景蒙版，去除一些噪声并使用形态学填充小间隙，然后找到连接的组件，也就是“斑点”。通常情况下，blob确实包含阴影。

如果你的阴影清除算法在一个单独的框架上运行，那么这是它的代码去的地方。

Answer 2

我觉得你的问题是在你的目标检测。您的代码使用的是imopen和imclose。但我认为你想使用imdilate和imerode

%% Detect Objects 

function [centroids, bboxes, mask] = detectObjects(frame) 

    % Detect foreground. 
    mask = obj.detector.step(frame); 

    % Apply morphological operations to remove noise and fill in holes. 

    %this makes the white space smaller to remove small dots/noise 
    mask = imerode(mask, strel('rectangle', [3,3])); 

    %this makes the white regions larger to restore the original size 
    mask = imclose(mask, strel('rectangle', [15, 15])); 

    mask = imfill(mask, 'holes'); 

    % Perform blob analysis to find connected components. 
    [~, centroids, bboxes] = obj.blobAnalyser.step(mask); 
end 

我改变了你的imopen和imclose你可能需要改变你的结构元素。如果你看一下matlab documentation imopen是一样的侵蚀之后扩张，或imopened = imdilate(imerode(im,se))); Imclose其次是糜烂或imcloseed = imerode(imdilate(im,se))

扩张纸张使用侵蚀后跟一个扩张（一IMOPEN操作）。这就是我在代码中所做的。如果你要使用这两个业务相同的结构元素，你可以改变

%this makes the white space smaller to remove small dots/noise 
    mask = imerode(mask, strel('rectangle', [3,3])); 

    %this makes the white regions larger to restore the original size 
    mask = imclose(mask, strel('rectangle', [15, 15])); 

到

mask = imopen(mask,strel('rectangle',[15 15])); 

，因为你不包括你的照片，我不能尝试一下，但我认为这应该在你玩strel函数后工作