BIGDATAMEANS BIGOPPORTUNITIES FORTELEVISION ANDYAFTELAK VICEPRESIDENTANDDIRECTORFOR THEARRISAPPLIEDRESEARCHCENTER BHAVANGANDHI SENIORDIRECTOR,ANALYTICS&INSIGHT FORTHEARRISAPPLIEDRESEARCHCENTER
TABLEOFCONTENTS INTRODUCTION 3 DATAISEVERYWHERE 3 MAKINGDATAACTIONABLE 5 BIGDATAINACTION 6 APPLYINGBIGDATATO NDVR 8 CONCLUSION 10 MEETOUREXPERT:ANDREW AFTELAK 11 MEETOUREXPERT:BHAVANGANDHI 12 REFERENCES 12 Copyright2015 ARRISEnterprises,Inc.Alrightsreserved. 2
INTRODUCTION Theexperienceofwatchingtelevisionisevolvingfast.Throughthecourseofafew shortyears, we veseenchangesinthemethodsthroughwhichtelevisionprogrammingisdistributed,and thewaysinwhichitisconsumed.we vegonefrom ascheduled,broadcastmodeltooneof on-demand,time-shiftedvideodelivery.we veseentelevisionbreakfreeofthelivingroom TV andfinditshomeonawiderangeofdevicesthatcanbetakenvirtualyanywhere.andwe ve seentvviewingbehaviorschangefrom thepassive leanback experiencesofyesterday,to today smoreinteractive lean-in viewing. Yetwhilethedeliveryandconsumptionoftelevisionprogramminghaschangedindiverse ways,onefactorisemergingtobindthem altogether.eachofthesechangesbringswithita new opportunitytogleancriticalinformationonservicedelivery,viewingbehaviorsandthe contentitself.thisistheageofbigdataenabledtelevision. Asweenterthisnew era,thereareseveralopportunitiesforserviceproviderstoexploitthe manydatapointsthatarenow availabletothem.thekeyistodeterminewhichinformation ismostimportant,andhow tobestaggregate,analyzeandactonittoimprovetheuser experience,increasesystem efficiencyandmakemoreprofitablecontentdecisions.this paperintroducesseveralopportunitiesthatawaitserviceprovidersattheintersectionof televisionandbigdata,andprovideskeyinsightsthatcanhelpthem realizetheirnear-and long-term businessobjectives. DATAISEVERYWHERE Therearenumeroussourcesofdatawithinthetelevisiondeliveryenvironment,andinorder toproduceactionableresults,itmustbecolectedinanumberofkeylocationsateverylayer ofservicedelivery.aseachpieceofcontentisdeliveredfrom itssourcetothecustomer premises,ittouchesseveralinfrastructureplatformsalongtheway-from encoderstothe accessnetwork,althewaytoset-topboxesandenduserdevices.serviceprovidersmustbe abletocapturecriticaldataateachofthesepointsalongthecontentjourney.andthat requiresinstrumentationtobebuiltintothemanyhardwaredevicesandsoftwarecompon- entsinthetelevisiondeliveryecosystem. 3 WEB BLOG www.arris.com www.arriseverywhere.com
SOURCE ENCODERS HEADEND CORE CPE 1:Instrumentationisrequiredacrostheend-to-endtelevisiondeliveryecosystem Inadditiontocapturingdataonahostofphysicaldevices,serviceprovidersmustalsoapply instrumentationinthelogicalrealm -atthenetwork,videodelivery,controlandapplication layers.thismeansextractingdataonthequalityofvideostreams,understandingback-office operationalsystems,andapplyingdatacolectiontothemanyinterfacesavailableto subscribers. USER Application:Clientapplicationinstrumentation,userprofiling&targeting Control:QoS-Fulfiliment,OSS(nDVR,advertising,etc.) VideoDelivery:REC,JITP,QoV&QoE-VisualQualityoftheVideo(PSNR,SSIM,MOS,etc.) Network:QoS-networkstatistics,jiter,packetlevelinfo 2:Datacanbecolectedateverylayeroftheservicedeliveryenvironment Aholisticapproachtodatacaptureensuresthateveryimportantdynamicrelatedtothe system,usersandcontentcanbetreatedasaviableinformationsource.thisrequiresthe colectionofanabundanceofdata,includinginterestsinprograms,genresandactors; recordingandplaybackbehaviors;viewingtimesandsessionlengths;multiscreendevice types;streamingbandwidthconsumption;infrastructureperformance;andinterfaceactivities. Thereisalsoanopportunitytocolectdatabasedonthedynamicsofthecontentbeing delivered.thisincludesscenebreaks,scenecuts,motionandfaces,justtonameafew.in addition,contentisnow appendedwithrichdataintheelectronicprogram guideonplot lines,actors,genresandratings.andformostcontent,awritentranscriptisavailableinthe form ofclosedcaptioning,whichprovidesdetailedcluesastowhat sgoingonatagiven pointintime.inordertogainthemostvaluefrom BigData,therightinformationneedsto becolectedfrom withintheservicedeliveryenvironment. Copyright2015 ARRISEnterprises,Inc.Alrightsreserved. 4
Butcontentspecificsanddatafrom theservicedeliveryenvironmentalonemaynotbe enough.totrulyunderstandthedynamicsoftheirprogramming,serviceprovidersmust colectdatafrom theoutsideworld.thismeansextractingkeyinformationfrom theworld ofsocialmedia.thiscantaketheform ofkeywordsthataretrending,spikesintrafficduring livebroadcastsandcontentfrom industryinfluencers.byunderstandingwhattheoutside worldissayingabouttelevisioncontent,serviceproviderscancompletethecomprehensive picturetheyneedtoapplydeepmeaningtoon-screencontent,andmoreimportantly to actonthisinsight. MAKINGDATAACTIONABLE Oncetherightdatahasbeencolected,itiscrucialtobeabletomakesenseofital,withthe goaloftakingaction.thisrequiresamodern,bigdataapproachtothecolection,organization andanalysisofinformation.ratherthandeterminingkeyperformanceindicators(kpis)and capturingonlythethresholdsassociatedwiththem,serviceprovidersmustinsteadcolectand storeraw dataforeventsastheyhappen.thisenablesbothreal-timeandbatchanalysis, whichform thefoundationforfast,inteligentdecisions,andalsoalow ahistoricalviewpoint toimprovelong-rangeplanning. Thedatamaythenbestructuredandorganizedwithinintermediatetablesthatmakesense oftheabundantdatapointsthatarecolected.toturnthiswealthofraw dataintoactionable information,aclearandcustomizablereportingsystem isneeded.thesereportsprovidedetails Programswatched. Channelbrowsed. Adswatched,skipped. Recommendationfolowed. Subscriberdetails. HadoopHDFS Raw Datastreams Spark Real-time processing Spark Batchprocessing forhistorical trendanalysis PostGres Marketsegmentation. Intermediate Tables Batchprocessing Scalding/Spark Real-time processingfor currentactivity reports HIVEReport Tables Reportcreation BusinessInteligenceDashboards 3:TheARRISarchitecturefortelevisionanalyticsandinsight 5 WEB www.arris.com BLOG www.arriseverywhere.com
onbothcurentactivityandhistoricaltrends.finaly,bylayeringtherightinteligenceontop ofthesereports,itbecomespossibletovisualizethedataandanalyzeitinanumberofuseful ways.withabusinessinteligencedashboard,serviceproviderscancrossreferencemultiple datapointstoderivedeepmeaningabouttheirservices,subscribersandcontent.anexample ofthecomprehensivebigdataarchitecturefortelevisioncanbefoundin3.andis 1 exploredingreaterdepthinarecentarristechnicalpaper. Inordertoactonthisimportantdata,serviceproviderscanadjusttheirservicedeliveryecosystemsmanualy,makingchangeswithintheirhardwareandsoftwareinfrastructureto optimizeperformanceorbetermeettheircustomers needs.inmoreadvancedimplementations,serviceproviderscanautomatethesechangesbyleveragingapowerfulapilayerthat canshareinformationwithotherservices,buildingnew levelsofresponsivenessintotheir systems.inthesectionthatfolows,wereview someofthemanywaysthatserviceproviders canactondatatoimprovetelevisiondelivery. BIGDATAINACTION AtthedawnoftheBigDatatelevisionera,itiscriticalforserviceproviderstoprioritizetheir information-relatedobjectives.thesetypicalyfalintothreecategories:optimizingthe system,enhancingthecustomerexperience,andimprovingtheperformanceofcontent. ByapplyingaBigDataapproachacrosstheseobjectives,serviceproviderscanachieve valuablegains. Whenitcomestothetelevisiondeliverysystem,everydolarcounts.Soitiscriticalfor serviceproviderstogetthemostoutoftheircapitalandoperationalinvestments andbig Dataprovidesasignificantopportunitytodoso.Bycolectingandanalyzingtherightinformation,serviceproviderscanunderstandhow theirsystemsareoperatingovervirtualyany periodoftime.forexample,serviceproviderscandeterminethatstreamingresourcesare beingstrainedatcertaintimesofdayorthatstorageresourcesarenearingathresholdof maximum utilization.theymightalsousedatatoderivethattheirnetworkisnotreadyto carryultrahd4kcontent.thisinsightintosystem performancecanhelpthem morequickly identifyandresolveserviceissues,morepreciselyfine-tunetheiroperationalsystems,and moreaccuratelydeterminewherecapitalinvestmentsareneeded. BigDatacanalsohelpimprovethecustomerexperience.Withnew dataonhow subscribers areinteractingwithcontent,serviceproviderscanbeterhonetheiruserinterfacesand overalserviceenvironmentstomatchtheircustomers preferences.byunderstandingthe uptakeofnew features,theycanidentifynew opportunitiesforinvestmentormodifytheir Copyright2015 ARRISEnterprises,Inc.Alrightsreserved. 6
Throughadvancedmediaanalysistechniques,serviceproviderscandeterminewhenkey playsoccurduringsportsbroadcastsorwhererelevantsegmentsofanewsbroadcastcan befound.thislevelofcontent-levelanalysiscanform thefoundationfornew contentcuration offerings,whereshortsegmentsfrom multipleprogramscanbepackagedaroundspecific userpreferences. Contentinteligencecanalsohelpserviceprovidersascertainwhereadbreaksreside-even intheabsenceofscte-35markers.byunderstandingwhenthesebreaksoccur,service providerscaninsertalternativeadvertisementsinrecordedprogrammingbeyondthec3or C7adwindow.Thiscanopennew revenuestreamsbasedonexistingcontent,especialy whentheadsaretargetedtospecificusersorhouseholdsbasedonananalysisofusers previouscontentchoices. offeringstoincreaseusage.thiscanapplytofeatureswithintheprogram guide,dvrservice, multiscreeninterfaceoron-demandoffering.thisnew insightcanhelpyieldvaluabletargetedadvertisingandmarketingopportunitiesbasedonuserpreferenceandbehavior. Withnewfoundvisibilityintousers preferencesforandinteractionswithtelevisionprogramming,serviceproviderscanmoreaccuratelyassignvaluetothecontenttheypurchase.and thatmeanstheyareinastrongerpositionwhennegotiatingcontentagreementswithprogrammers.onceserviceprovidersknow how frequentlytheirusersaretuningin,recording, playingback,andstreamingapieceofcontent,theycanbegintoascribeanaccuratevalue toit,andgainnew insightsintotheprofitabilityoftheirprogrammingonamuchdeeperlevel. Whencombined,thebenefitsofBigDatacanadduptomorethanoptimizedsystems,enhancedexperiencesandimprovedcontentperformancealone.Evenmorepowerfulopportunitiesarisewhenmultipledatasetsarestitchedtogethertocreatehigh-valuecrossreferences. TheARRISarchitectureforthisadvancedanalysis,includinganadvancedMediaAnalysis Framework(MAF),isshownin4. UserProfiles MAF Appliance Services Metadata Database Database andweb Services Offer Management and Recommendation Application Backendand Guide Services Clients ContentSources (live,vod) MAF 4:TheARRIS frameworkforadvancedanalysis SocialTrending ClientandContent Delivery Instrumentation 7 WEB BLOG www.arris.com www.arriseverywhere.com
Bydrawingconnectionsbetweenthedataassociatedwithsystem performance,useractivity andsocialmediainsight,serviceproviderscanbegintouncoverthehiddenopportunities behindtheirprogramming.forexample,ahistoricalanalysisofthesefactorsmayreveala corelationbetweenatrendingtopicontwiter,aspikeinviewingactivity,andanoverloaded streamingserver.thistypeofanalysishasthepotentialtoenableserviceproviderstofindnew, moreproactiveindicatorsofpendingperformanceissuesbyusingbigdataininnovativenew ways. APPLYINGBIGDATATO NDVR Inarecentstudyof26,000subscribersusingareal-worldnDVRplatform,ARRISwasableto gainkeyinsightsthatrepresentvaluableopportunitiesforserviceproviderstoimprovetheir videoofferings.asampleofthedatarevealedthat-notsurprisingly-mostviewerswerefast forwardingthroughadvertisementsduringagivenprogram,asindicatedbytheuniquefingerprintofascenechangeprecedingaspikeinfastforwardcommandsasmeasuredacrossthe set-topboxesinthesample. Evenmoreinterestingly,most oftheseuserswerefoundto befastforwardingbeyondthe advertisementsandintothe programming,forcingthem to rewindinsearchofthebegin- ningofthescene.manyofthese userswereunabletotimetheir playbackwiththestartofthe scene,andendedupwatching theendofthelastadvertisement beforetheprogram resumed. Thistypeofanalysiscanhelp serviceprovidersandprogram- mersassignmoreaccurate pricingtothevarioussegments ofanadbreak. TRICKPLAYUSAGE No.ofUsers 70. 60. 50. 40. 30. 20. 10. 0 00:00 00:08 Ad/Program Detected Program CONTENTANALYSIS Ad. 00:00 00:08 Play FFwd Rewind TIME TIME 5:Sampledataonfastforwardandrewindbehaviors Copyright2015 ARRISEnterprises,Inc.Alrightsreserved. 8
6: Comparing recordingto playback behaviorsover time UserActivity Overview 65K 60K 55K 50K 45K 40K 35K 30K 25K 20K 15K 10K 5K 0 CABLEOPERATORSAMPLEOF26,000USERS Recordingbehavioriscyclical; Recordingsareplayedbackuniformly DVRPlayback DVRRecordings VODPlays Jan3 Jan8 Jan13 Jan18 Jan23 Jan28 Feb2 Feb7 Feb12 Feb17 (Date2014) Dataanalysisfurthershowedthatwhilerecordingbehaviorishighlyvariable,playbackisfar moreuniform.thiscanhelpserviceprovidersdeterminethebestapplicationsforvirtualized softwareanddedicatedhardwareinfrastructure.inaddition,theanalysisestablisheda paternforpeakrecordingandplaybackbehaviors,depictingthesteadygrowthinthese activitiesovermultiplemonths.thislevelofinsightalowsserviceproviderstobeterplan theirnext-generationofferings,alocatingtheproperstorageandrecordingresourcesto ensuretheyarealignedwithsubscriberneeds.furthermore,thisanalysishelpstheservice providerpredictitscapacityneedsbasedonactualcontentconsumptionandserviceusage. PeakActivity 80K OPERATIONALINSIGHTFROM USAGE 70K 60K 50K 40K 30K 7: Comparing peakrecording andplayback behaviors 20K 10K 0 Jan4 Jan14 Jan24 Feb3 Feb13 Feb23 Mar5 Mar15 Mar25 Apr4 Apr14 Apri24 May4 May14 Max-PeakRecordingbehavior Max-PeakPlaybackbehavior 9 WEB www.arris.com BLOG www.arriseverywhere.com
Finaly,thedatashowedthepaternofsubscribersrecordingcontentandviewingitwithin threedays,establishingalinkagebetweenthetypeofcontentandusertendenciestoview it quickly.theanalysisindicatesthepopularityofcontentonamuchdeeperlevelthanhasbeen historicalypossible,assigningvalueforthevariouscontenttypesintheirlibrariesandbundles. Inthiscase,thedataindicatedastrongpreferenceformajorliveevents. BasedonthenDVRexampleandthewindow intoactualcontentbehaviorthatbigdata presents,serviceprovidersareabletomonetizetheircontentmoreeffectivelythanever presents,serviceprovidersareabletomonetizetheircontentmoreeffectivelythanever before.butthisexampleonlyscratchesthesurfaceofwhatispossiblewhenbigdataand televisionunite. CONCLUSION AsBigDatatakesonagreaterroleinthedeliveryoftelevisionservices,thepossibilitiesfor optimizingthesubscriberexperiencearevirtualylimitless.inthispaper,we vereviewed severalexamplesofthemanybenefitsserviceproviderscanrealizewhentherightinformation iscolected,andtherightsystemsforanalyzingdataaredeployed.whetheritisimproving iscolected,andtherightsystemsforanalyzingdataaredeployed.whetheritisimproving therelevanceofadvertising,remonetizingcontentinnew ways,proactivelyidentifying potentialserviceissuesorbeterunderstandingthevalueofcontent,bigdatacanmeanbig gainswhenitcomestotelevisiondelivery. Denver Broncos Vs.SeaH Opening Ceremony Olympics Snow boarding Snow boarding Madeline Cross Country Sking Modern Family Freestyle Sking Short Track Alpine Sking Biatholon DVRPlayback DVRPlaysinC72Window Recording UniqueUsers CustomersRecordingcontentandviewingwithin3days Majorliveeventsatractaboveaverageviewershipforchannel Program Popularity CABLEOPERATORSAMPLEOF26,000USERS 8:Comparingrecordingandplaybackactivitiesformultipleprograms 10 Copyright2015 ARRISEnterprises,Inc.Alrightsreserved.
Whenitcomestoenablingthisnew eraofinformation,thekeyforserviceprovidersistothink big.ratherthancolectingasliverofdatabasedonalimitednumberofeventsandthresholds, serviceprovidersmustgather,storeandstructureanabundanceofraw informationfrom withintheservicedeliveryenvironmentandtheoutsideworld.insteadofchoosingbetween real-timeorbatchprocessing,serviceproviderscanbenefitbyselectingsystemsthatcan perform bothtypesofanalysis.andgetingthemostoutofdatameansdeployingsystems thatcanpresentinformationinanumberofusefulways,andevenapplyautomationto optimizesystemsinrealtime. PerhapsmostimportantlyistheneedtoapplytelevisiondomainexpertisetoBigData,seting strategiesandchoosingplatformsthatcanmaximizetheuniqueinformationthatcantruly improvetheexperienceforsubscribers.bydesigningabigdataapproachthatisbasedonthe uniqueneedsofthetelevisiondeliveryecosystem,serviceproviderscanextend thevaluethey delivertoendusers,driveoperationalefficienciesthatkeepcostsdown,andextractmore valuefrom theirvideocontent. MEETOUREXPERT:ANDREW AFTELAK MeetAndyAftelak,VicePresidentandDirectorof theappliedresearchcenteratarris.heleads theteam thatcreatesnew productsandservices thatarechangingthewaypeopleenjoymultimedia,bothathomeandonthemove.througha seriesoftechnicalleadershippositionsthathave spanneda20-yearcareer,andyhasconstantly leveragedhisexperiencewithmoderncommunicationssystems,humanfactorprocessesa nduser-centricresearchtoguidethedevelopment ofadvancedtechnologies.hereceivedhis DoctorateandBachelor'sdegreesinElectronic andelectricalengineeringfrom Loughborough UniversityinEngland.Afelow oftheinstitutionof EngineeringandTechnology,Andyholds23 patentsandhasbeenactivelyinvolvedin internationalstandardsbodiesandindustry forums. 11 WEB BLOG www.arris.com www.arriseverywhere.com
MEETOUREXPERT:BHAVANGANDHI MeetBhavanGandhi,SeniorDirector,Analytics& InsightfortheAppliedResearchCenteratARRIS. Heleadsamulti-disciplinaryteam ofresearchers whoprototypenew interactiveandconverged mediaapplicationsforthehomeandmobile markets.bhavanisacloudservicesandvideo expertwhohassuccessfulytakenmedia-centric technologiesfrom R&Dtoproductimpact.Partof histeam smissionistounderstandhow video deliverysystemscanbeinstrumentedtocolect actionabledataandanalyticsthatcanprovide moremeaningfuluserinteractionanddrive networkefficiencies.bhavaniswelversedin imageandvideo-processing,aswelasbigdata analytics,with18issuedpatents.hereceivedb.s. andm.s.degreesinelectricalengineeringfrom theuniversityofilinoisaturbana-champaign. REFERENCES 1.Ghandi,B.(2015).TVInsights ApplicationofBigDatatoTelevision. ARRISEnterprises,Inc.2015Alrightsreserved.Nopartofthispublicationmaybereproducedinanyform orbyany meansorusedtomakeanyderivativework(suchastranslation,transformation,oradaptation)withoutwriten permissionfrom ARRISEnterprises,Inc.( ARRIS ).ARRISreservestherighttorevisethispublicationandtomake changesincontentfrom timetotimewithoutobligationonthepartofarristoprovidenotificationofsuchrevision orchange. Copyright2015 ARRISEnterprises,Inc.Alrightsreserved. 12