自制精简版眼动仪

Starburst Arobustalgorithmforvideo basedeyetrackingDonghengLi DerrickJ ParkhurstHumanComputerInteractionProgramIowaStateUniversity Ames Iowa g9517鄭聖耀g9518王凱平g9524林裕凱 Outline 前言自製眼動儀介紹實作方法瞳孔偵測原理凝視點的投射原理實驗結果結論 前言 1 2 眼動儀歷史 accordingtowikipedia 1800s 以直接觀察的方式測量視線軌跡1879 Louis mileJaval 眼科醫師 提出閱讀時 視線移動是由一連串的凝視 fixation 及掃視 saccade 所組合的行為模式1897 EdmundBurkeHuey製作第一台眼動儀 侵入式 1935 GeorgeBuswell製作第一台非侵入式眼動儀1950s AlfredL Yarbus 心理學家 確立了凝視點的移動與實驗素材的內容有絕對的影響 1980 JustandCarpenter 語言學家 提出了凝視點代表思考的假說 前言 2 2 眼動儀相關知識Video basedeyetrackers瞳孔 pupil 中心角膜反射點cornealreflection CR BrightPupil光源直接照射瞳孔DarkPupil光源稍微偏離瞳孔取樣頻率至少30Hz 目前技術最大可達到1250Hz閱讀文章的凝視點平均時間是200ms影像的平均凝視點平均時間是350ms準備掃視到下一個目標大約是200msActive passiveinterfaces Outline 前言自製眼動儀介紹實作方法瞳孔偵測原理凝視點的投射原理實驗結果結論 自製眼動儀介紹 1 2 自製眼動儀介紹 2 2 與iViewXHi Speed之比較 Outline 前言自製眼動儀介紹實作方法瞳孔偵測原理凝視點的投射原理實驗結果結論 實作方法 1 15 Feature based從影像中找出可能是眼睛的特徵點飽和度門檻值 intensitythreshold 異色 limbus 邊緣特徵點的幾何圖形 橢圓形 正方形 Model based找出圖形的最佳參數提供更準確的圖形需有一個好的初始值 實作方法 2 15 Starburstalgorithm1Input Eyeimage Sceneimage2Output Pointofgaze3Procedure 4Detectthecornealreflection5Localizethecornealreflection6Removethecornealreflection7Iterativedetectionofcandidatefeaturepoints8ApplyRANSACtofindfeaturepointconsensusset9Determinebest fittingellipseusingconsensusset10Model basedoptimizationofellipseparameters11Applycalibrationtoestimatepointofgaze 實作方法 3 15 NoiseReduction optional shotnoise5 5Gaussianfilterlinenoise調整每列的飽和度至原來的平均值Cornealreflectiondetection localizationandremoval以動態門檻值找出eyeimage中最亮的部份從最大值開始遞減 計算最大的區塊面積與其他區塊的總面積之比值當比值開始降低就結束以sqrt Area pi 決定半徑移除最亮區域 實作方法 4 15 動態threshold示意圖 目標 實作方法 5 15 Starburstalgorithm1Input Eyeimage Sceneimage2Output Pointofgaze3Procedure 4Detectthecornealreflection5Localizethecornealreflection6Removethecornealreflection7Iterativedetectionofcandidatefeaturepoints8ApplyRANSACtofindfeaturepointconsensusset9Determinebest fittingellipseusingconsensusset10Model basedoptimizationofellipseparameters11Applycalibrationtoestimatepointofgaze 實作方法 6 15 PupilcontourdetectionFeature detectionalgorithm1Input Eyeimagewithcornealreflectionremoved Bestguessofpupilcenter2Output Setoffeaturepoints3Procedure 4IterateStage1 FollowraysextendingfromthestartingpointCalculateintensityderivativeateachpointIfderivative thresholdthenPlacefeaturepointHaltmarchalongrayStage2 ForeachfeaturepointdetectedinStage1MarchalongraysreturningtowardsthestartingpointCalculateintensityderivativeateachpointIfderivative thresholdthenPlacefeaturepointHaltmarchalongrayStartingpoint geometriccenteroffeaturepoints19Untilstartingpointconverges 實作方法 7 15 實作方法 8 15 Starburstalgorithm1Input Eyeimage Sceneimage2Output Pointofgaze3Procedure 4Detectthecornealreflection5Localizethecornealreflection6Removethecornealreflection7Iterativedetectionofcandidatefeaturepoints8ApplyRANSACtofindfeaturepointconsensusset9Determinebest fittingellipseusingconsensusset10Model basedoptimizationofellipseparameters11Applycalibrationtoestimatepointofgaze 實作方法 9 15 Ellipsefitting從featurepoints猜一個橢圓形以RandomSampleConsensus RANSAC 求得最佳橢圓形Fit EllipseRANSACalgorithm1Input Featurepoints ransac iteration largenumber Output bestfittingellipseparametersProcedure Random 0IterationIfRandom 0thenPick5differentrandompointsfromfeaturepointsElseFindallpointsfitwithinformerellipsemodelUsethemtogenerateanewellipsemodelIfthismodelisgoodenoughRandom 1Ifthisellipsemodelis better N log 1 0 99 log 1 W5 UntilN ransac iteration 實作方法 10 15 better 這個model對每個featurepoint錯誤越少圓錐曲線方程式ax2 bxy cy2 dx ey f 0以SVD求齊次方程式解 homogeneoussolution 齊次解轉換成矩陣H利用來計算每個點誤差值 是 x y 向量W fitellipse的點的個數 所有featurepoint的個數 1 W5 N 從所有feature點中連挑5個點 每個都沒有fitellipse 重複N次之下的機率設正確率為99 1 0 99 1 W5 N 希望在N次之下錯誤率很低N log 1 0 99 log 1 W5 同取log以求得N 實作方法 11 15 實作方法 12 15 Starburstalgorithm1Input Eyeimage Sceneimage2Output Pointofgaze3Procedure 4Detectthecornealreflection5Localizethecornealreflection6Removethecornealreflection7Iterativedetectionofcandidatefeaturepoints8ApplyRANSACtofindfeaturepointconsensusset9Determinebest fittingellipseusingconsensusset10Model basedoptimizationofellipseparameters11Applycalibrationtoestimatepointofgaze Outline 前言自製眼動儀介紹實作方法瞳孔偵測原理凝視點的投射原理實驗結果結論 實作方法 13 15 SingularValueDecomposition SVD solveleast squaresminimizationSeekthatminimizes Letand fori 1 n Finally 實作方法 14 15 SVDsolvehomogeneousequationsFindthethatminimizessubjecttominimize andLet minimizesubjectto因為D是對角矩陣 且對角值由上到下是從大到小排列 y 0 0 1 T Finally 所以是V的最後一行向量 實作方法 15 15 Calibration測量兩張影像中相對應的9對點計算最佳的對應 產生係數 Linear以SVD做最小誤差2nd orderpolynomial3rd orderpolynomialHomographic以SVD計算homographic矩陣H Outlin