Teknik og implementeringsmøde 13. November 2013 Dato - Dok.nr. 1
Data kategorier der skal migreres 4 kategorier: 1. Pris informationer - Prisliste 2. Tilknytning af tariffer, abonnementer pr målepunkt 3. Stamdata indikator for kunder med elvarme 4. Andre stamdata Elleverandør stamdata Nye målepunkts stamdata De første 3 kategorier skal være tilstede for at elleverandøren kan fakturerer kunderne korrekt Dato - Dok.nr. 2
Migration - data and responsibility Until 1/10/2014 After 1/10/2014 Migration Migration Operation (B2B only) Data elements Sender Recipient How Sender Recipient How Sender Recipient 1. Price elements GO/ ENDK DH GUI DH BS B2B GO BS 2. Grid tariffs and subscription links GO/ ENDK DH File 1 DH BS B2B/ File 3. TAX/ENDK links GO DH File 1 DH BS B2B/ File 4. Electrical heating GO DH File 2 DH BS B2B/ File 5. Contact info GO DH File 3 DH BS B2B/ File 6. New GO MP detail (meter details + peak limit) GO BS BS BS BS DH DH GO - GO BS B2B GO BS 7. Consumer info - BS DH B2B BS DH 8. Reading of 01-10-2014 (BRS-020) 9. Existing data with changed owner - GO BS B2B GO BS GO BS B2B BS DH 10. Existing data with no changes GO BS B2B GO BS Cut-over Dato - Dok.nr. 3
Migrering fra netvirksomhed til datahub Netvirksomhed Datahub DH 1 Produktion Dato - Dok.nr. 4
Migrering fra netvirksomhed til datahub Netvirksomhed Datahub Type 1 Type 2 Type 3 Filtype 1 Tilknytninger (links) Fuld liste med alle MP er E17 og E18 Filtype 2 Kunder med elvarme Positiv liste Filtype 3 Kontaktinformation DH 1 Aflæsnings-, valg- og afbryderkort Produktion Positiv liste Dato - Dok.nr. 5
Migrering fra datahub til elleverandør Datahub Elleverandør DH 1 Produktion B2B Dato - Dok.nr. 6
Migreringscirklen fra netvirksomhed til datahub Verificer Load Type 1,2,3 Type 1,2,3 Modtag Afsend Delta Opdater Dato - Dok.nr. 7
Fuld dataset Fuld dataset Fuld dataset Fuld dataset Fuld dataset Successiv data migrering Initiel 2 upload 3 upload n upload Final NV Delta Efter final delta bliver data fixeret til system go-live Datahub production DH DQ reporting DQ reporting DQ reporting DQ reporting Dato - Dok.nr. 8
Delta beregning i hovedtræk Filtype 1 - Tilknytninger Delta beregningen foretages på alle felter - Fuldstændigt match MP i migreringsfil, men ikke i DH-links tilknytning oprettes MP ikke i migreringsfil, men i DH-links tilknytning slettes MP i både migreringsfil og DH-links, men data forskellig tilknytning opdateres Filtype 2 - Elvarme Udgangspunktet ved opstart af migrering, er at alle elvarme flag er sat til 0 Ingen elvarme Delta beregningen foretages på baggrund af målepunkter i migreringsfil De målepunkter der findes i migreringsfilen får sat elvarme flag til 1 Elvarme Målepunkter der ikke findes i migreringsfil vil blive sat til 0 Ingen elvarme Filtype 3 - Kontaktinformation Delta beregning foretages på hele recorden hvis feltet SameAsMPAddress er sat til 0 (falsk) Alle felter i recorden bruges databasefelter overskrives med nye værdier fra migreringsfil Delta beregning foretages på dele af recorden hvis feltet SameAsMPAddress er sat til 1 (sandt) Kun felterne (Adressetype, Kontaktnavn1, Kontaktnavn2) bruges fra recorden alle resterende felter tømmes i databasen Dato - Dok.nr. 9
Output from migration process to actors Energinet.dk DataTeam SMS migration status Migration status DataTeam member comment Number of delta records Number of rejected records Migration report Delta records FTP Rejected records Migration status DataTeam member comment Number of source records Number of delta records for insert Number of delta records for update Number of delta records for delete Meteringpoints with no relation Dato - Dok.nr. 10
Migrationdata from grid operator to Datahub Grid operator Timestamp # records CRC value Filename Filesize File metadata Staging Staging Scheduling/ Verification DH data team delta FTP File verification Staging loader Delta calculator Load tool Failure Visual spot check Failure Failure Failure ODS CCR IS Change Success Failure / Rejected meteringpoints Migration path for data elements in category 2,3,4 and 5 Valid until 1. oct 2014 Datahub Production Dato - Dok.nr. 11
Migrationdata from Datahub to Balance Supplier Balance Supplier BRS-005 Datahub Production BRS-034 BRS-038 B2B (normal operation) Balance Supplier Balance Supplier Balance Supplier ODS Data Extraction FTP Balance Supplier Balance Supplier Tariff/Subscription/Fee links Electrical heating Contact Data Distribute DH data team Balance suppliers selected by Energinet.dk Migration path for data elements in category 2,3,4 and 5 Valid from 1. oct 2014 Dato - Dok.nr. 12
Migration utility status codes Receiving csv file Verify file metadata Format & data check before staging load Spot check validation Calculating delta Scheduling migration Loading delta using B2B FTP File verification Staging loader Delta calculator Datahub loader Internal status Received Verified Loaded Validated Calculated Scheduled Migrated External status Rejected Failed Failed Warning Failed/succeded Migration report Rejected records Dato - Dok.nr. 13
Web user interface for data support Search Migration Files GridOperator Include Previously Processed Files ProcessingStatus FromDate ToDate Search Clear Files File Organisation File Date File Size Nr of Lines Status Filename <1> Filename <2> Filename <3> OrganisationName File Date File Size Lines Status Start Validation Reject OrganisationName File Date File Size Lines Status Start Import Reject OrganisationName File Date File Size Lines Status Start Load Delta Reject Filename <N> OrganisationName File Date File Size Lines Status <Next Step> Reject Compare selected 145 Results Previous 1...3 4 5 6 7 8 9...15 Next Page size 10 Dato - Dok.nr. 14
File format 1 Tariff and subscription links (Full list) Pre-requisite Price definitions is created using GUI or B2B Price elements must have been defined before linking of tariffs and subscriptions Input fields MeteringPointId ValidFromDate ValidToDate ChargeType ChargeTypeCode ChargeTypeOwner Quantity N18 D10 D10 A1 A10 N13 N3 Attributes description ChargeType = S (subscription) or T (tariff) ChargeTypeCode is a logical code related to tariff or subscription as defined in the price element definition ChargeTypeOwner is GLN number Quantity is number of subscriptions. Attribute not relevant for tariffs (always 1 for tariffs) If ValidToDate < 01.10.2014 then skip Dato - Dok.nr. 15
Felttype validering File format 1 Felt Felttype Størrelse Værdi Meteringpoint Nummerisk Fixed længde = 18 Kun tal Tal og - Valid From Date Dato Fixed længde = 10 Format: YYYY-MM-DD Dato skal være 01.10.2014 eller senere Tal og - Valid To Date Dato Fixed længde = 10 Format: YYYY-MM-DD Dato skal være senere end Valid From Date Charge Type Alfanumerisk Fixed længde = 1 T eller S Charge Type Code Alfanumerisk Længde <= 10 Bestemt af elleverandør Charge Type Owner Numerisk Fixed længde = 13 GLN nummer Samme som upload folder Størrelsesregel gælder kun for ChargeType = S Quantity Numerisk 0 >= Længde <= 3 Værdi bruges ikke hvis ChargeType = T Dato - Dok.nr. 16
File format 2 Electrical heating (+ list) Pre-requisite Default TaxReduction is set to False for all relevant metering points TaxSettlementDate is emptied Input fields MeteringPointId N18 ValidFromDate D10 Attributes set in datahub TaxReduction set to True for the given MeteringPoint If ValidFromDate > 01-10-2014 TaxSettlementDate is set to ValidFromDate otherwise set to 01-10-2014 Dato - Dok.nr. 17
Felttype validering File format 2 Felt Felttype Størrelse Værdi Meteringpoint Nummerisk Fixed længde = 18 Kun tal Tal og - Valid From Date Dato Fixed længde = 10 Format: YYYY-MM-DD Hvis dato er før 01.10.2014 sættes datoen til den 01.10.2014 Dato - Dok.nr. 18
File format 3 Contact information (+ list) Pre-requisite AddressTypes must have been set up per grid area (configuration created by Energinet.dk based on input from Grid Operators) Input fields... MeteringP ointid Address Type Contact Name 1 Contact Name 2 Same As MP Address Street Code Street Name Building Number Floor Id N18 A3 A132 A132 B1 N4 A40 A6 A2... continued Room Id Post code City Name City Sub Division Name Municipality Code Country Code Email Address Phone Number Mobile Number A4 A10 A25 A25 A3 A2 A60 A20 A20 Attributes set in datahub SameAsMPAddress = 0 (No) or 1 (Yes) All or mandatory input fields from the delta Dato - Dok.nr. 19
Felttype validering File format 3 Felt Felttype Størrelse Værdi Meteringpoint Nummerisk Fixed længde = 18 Kun tal Address Type Alfanumerisk Fixed længde = 3 Contact Name 1 Alfanumerisk 0 > Længde <= 132 Contact Name 2 Alfanumerisk 0 >= Længde <= 132 Kun 3 adressetyper: D01 = Afbryderkort D02 = Aflæsningskort D03 = Valgkort 123 Abc 123 Abc Same As MP Address Boolean (flag) Fixed længde = 1 0 eller 1 Street Code Street Name Building Number Floor Id Room Id Post code City Name Numerisk / Alfanumerisk 0 >= Længde <= felt længde 123 Abc City Sub Division Name Municipality Code Country Code Email Address Phone Number Mobile Number Dato - Dok.nr. 20
File example The separation symbol for the fields is the pipe-symbol ( ). A pipe-symbol within a field is not allowed. The load files must be ended by a carriage return / linefeed (CR/LF), (0D0A). Windows compatible The data of the load files may not contain linefeed-symbols (0A) The character encoding of the load files must be UTF-8 MeteringPointId ValidFromDate ValidToDate ChargeType ChargeTypeCode ChargeTypeOwner Quantity 578000000000000097 2014-10-01 2014-12-31 S 210SP003 5790000432752 12 578000000000000059 2014-10-01 2014-12-31 T 457TP005 5790000432752 1 578000000000000066 2014-10-01 2014-12-31 T 457TP006 5790000432752 45463 578000000000000073 2014-10-01 2014-12-31 S 210SP001 5790000432752 4 578000000000000080 2014-10-01 S 210SP002 5790000432752 1 578000000000000097 2014-10-01 2014-12-31 S 210SP003 5790000432752 12 578000000000000103 2014-10-01 2014-12-31 T 457TP007 5790000432752 0 578000000000000110 2014-10-01 2014-12-31 S 210SP004 5790000432752 356 Dato - Dok.nr. 21
Spørgsmål Henvendelser vedrørende data migration og cut-over sendes til: xstj@energinet.dk Dato - Dok.nr. 22