telecomHall.com | Hunter Blog | Tips Blog | Course Blog | Groups

Ericsson 4G XML OSS Parameters Dump Parser


#23

I simply have no words for these parsers you’re sharing!

The most amazing and spectacular job ever seen.

It help us a looooot!

Really, really Thank You!


#24

@Rafael_Caixeta, you are welcome.

A new version of the parser has been released. It fixes some issues around column misalignment and missing MOs. Everyone is strongly advised to use the new parser. Also, note that the old parser will/may not work after 1st August :). This we did so that members can get access to the new features and bug fixes in the new parsers.

@Guillermo_Gustavo_Go, let us know if this fixes the missing MOs you were having.

@Enir, the issue related to vsDataEUtranFreqRelation and vsDataEutranFreqRelation will be fixed in new release this week. We are still testing it. It is caused by the fact that Windows considers vsDataEUtranFreqRelation .csv and vsDataEutranFreqRelation.csv as the same files! We will have to rename one of the files. We will keep you updated.

@GrishManouk, changing the way the list values are separated is still being tested. We will let you know when this is ready.

Otherwise, enjoy and give us feedback if you run into any issues.

Emmanuel


#25

Hi @Emmanuel! We’re very pleased to see that someone made a tool to process those .xml of E//'s topology!

like @Guillermo_Gustavo_Go said before, some atributtes are structures that have several elements inside. Those elements are separated by commas… Is it there a way that as your .jar executable makes the .csv output sepparating atributtes by semicolon “;”? In this way, when we convert text to columns in excel columns dont shift.

Regards

Marcos


#26

Hi @Emmanuel! Many thanks to your excellent work!
I’ve faced with one problem when I importing data from different OSSs simultaneously. If OSSs have different releases some parameters with similar names have different registers in some characters. For example in one OSS we have ‘systemInformationBlock3_sNonIntraSearchv920Active’ but in another ‘systemInformationBlock3_sNonintraSearchv920Active’. This produces 2 columns in the resulted file and can’t be imported in SQL… Is it possible to add a switch to the parser to ignore register of a parameter name (all cups)?

Hi @Mgulius . I see no problem with structures in Excel - they are rounded by " .


#27

Hi @Emmanuel , how are u?

so, what about the new version of E/// Enodebs? Some MOs like OptionalFeatureLicense didn’t appear for these Equipments. Is this a missing information in the XML File or in the parser?


#28

@DSolo,

We will get back to you on this after deciding on the best way to handle this case. Thanks for the feedback.


#29

@Lucio,

This is probably an issue the XML files. The parser will handle XML for all software versions.


#30

The new equipment has a different MO structure. We don’t no if “Bulk CM Export” is exporting all MOs data.


#31

Hi @Emmanuel,
my problem is little bit more complex. At the end I need all data to be imported into Oracle database (11g on windows server). I use standard sqlldr for that.
Names of Ericsson parameters are often have more than 30 characters long. Solution - I did a mapping table in Oracle and dynamically create .ctl files with parameters sorted as they were in parsed files.
But… when I parse new xml the result will probably have different parameters in different positions (+ new or missed parameters depending on OSS version). Solution - I have to read first line in all resulted .txt files, create a list with all parameters (with PowerShell script), load the list into Oracle and create new .ctl files.
And it will be nice to have a unique autogenerated object_id (oid) and parent-child (oid, pid) relation in each file. It will significantly improve performance of postprocessing, reliability and simplify scripts (no need to remember all object relations types if we have 200+ tables).
Unfortunately I’m not a JAVA specialist but can try to help in SQL if you need…


#32

@DSolo,

Your problem is more complex indeed. I am aware of the character limitation in Oracle and other DBs. We are using PostgreSQL which has a longer default character length for the column names (63). So far this is sufficient for us.
Regarding your suggestion of creating unique object ids, I would like to understand it better. Please give a more detailed explanation with an example.

Note that the parsers are just a small ( but vital) part of what we are building. Still, we are aware that people have different uses for the data and use different DBs. Because of this we are mindful not to tie the implementation to any DB.

When it comes to helping, the more people contributing the faster we can deliver something useful
for the community. We are not only working with Java, VBScript, Bash, Python, SQL and any other languages that we find useful in the execution of whatever functionality we need. SQL in particular is very important for the post-processing so we would very much appreciate your help. Let’s follow this up in private message.

Emmanuel


#33

Hi Emmanuel,
excuse me for delayed response. Was too busy with urgent tasks…

I’m sure that in your main solution you have a common hierarchical table
with all referencing objects. Something like ctp_common_objects table in
Netact. In this table each object has its own object_id - integer or any
other hybrid key + referense to its parent (key in the same format).
In the XML export you do not have this IDs but can create them automaticaly

  • HASH or autoincremented integer value. So you do not need to add BSC_name
    and cell_name to neighbours table but just a link to this unique ID of
    the object (parent cell)… And it is very easy to create a small script
    and extract all these IDs and create the common table.

It will be nice if the parser will be little bit more intellectual. I mean
it can do some simple postprocessing : change names of extracted
parameters, filter parameters, filter tables, replace values. For example
we can have a file (or several files for each table) with simple
structure: Parameter name in source; Parameter name in export ( “-” if we
want to ignore this parameter); substitution rule (for example I want to
ignore non numeric characters).

  • the parser can create some statistics: number of parsed objects, tables
    and etc. Moreover it can extract names of all existed parameters in the xml
  • it will help to import results into SQL DB. I did a very simple
    powershell script for that (in the attachment), I think it can be
    interesting for others too.
    And one more thing - when parser find any mistake it should create a file
    (“bad”) with discription and cause of the problem. If SQL loader find this
    file it should stop loading and this will prevent loading of corrupted data.

BR,
Dmitry Solomatin

https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail
Без
вирусов. www.avast.ru
https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

2017-10-25 20:37 GMT+03:00 Emmanuel Robert Ssebaggala <admin@telecomhall.net


#34

@DSolo,

Thanks for suggestions. Sorry for the delayed response :slight_smile: I was abit busy too.

Concerning your suggestions:

  1. Adding child-parent reference keys. We had not given this much thought. We will implement, test and see if it should be part of the parser itself or some post-processing script.
  2. Add some statistics after parsing. We will add this to our to do list.
  3. We are planning to add a log file. It should have stats and any info related to errors.

Also, there is no attachment!

Emmanuel


#35

get_all_parameters.zip (563 Bytes)
Hi, here is a script to extract all parameters names from first rows of all exported tables.
This is my first powershell script and I had only 5 minutes to create it…


#36

I wonder what you can do with an hour :slight_smile: let me test it . Thanks


#37

Thank you, it’s so helpful for me.
but i did some tests and found something strange.
my CM sample file like below.
there are two xn:VsDataContainer ( LD10118M, LD10118K) and actually two are same object except only position of <es:cellId>1</es:cellId>.
but paser generated different “cellId” value of vsDataEUtranCellFDD.csv file. like this.
could you explain ?

– vsDataEUtranCellFDD.csv –

]vsDataEUtranCellFDD_id, cellId
LD10118M, 1
LD10118K, 1;2;2;2;2

===========================================================

        <xn:VsDataContainer id="LD10118M">
            <xn:attributes>
                <xn:vsDataType>vsDataEUtranCellFDD</xn:vsDataType>
                <xn:vsDataFormatVersion>EricssonSpecificAttributes.16.28</xn:vsDataFormatVersion>
                <es:vsDataEUtranCellFDD>
                <es:userLabel>LTE_LD10118_Sultanpur</es:userLabel>
                <es:frameStartOffset>
                <es:subFrameOffset>0</es:subFrameOffset>
                </es:frameStartOffset>
                <es:administrativeState>1</es:administrativeState>
                <es:earfcndl>1300</es:earfcndl>
                <es:earfcnul>19300</es:earfcnul>                    
                <es:physicalLayerCellIdGroup>138</es:physicalLayerCellIdGroup>
                <es:physicalLayerSubCellId>0</es:physicalLayerSubCellId>
                <es:pciConflict>2</es:pciConflict>
                <es:pciConflict>2</es:pciConflict>
                <es:pciConflict>3</es:pciConflict>
                <es:pciConflict>2</es:pciConflict>
                <es:pciConflict>2</es:pciConflict>
                <es:pciConflictCell>
                <es:enbId>915606</es:enbId>
                <es:cellId>2</es:cellId>
                <es:mcc>404</es:mcc>
                <es:mnc>10</es:mnc>
                <es:mncLength>2</es:mncLength>
                </es:pciConflictCell>
                <es:pciConflictCell>
                <es:enbId>915606</es:enbId>
                <es:cellId>2</es:cellId>
                <es:mcc>404</es:mcc>
                <es:mnc>10</es:mnc>
                <es:mncLength>2</es:mncLength>
                </es:pciConflictCell>
                <es:pciConflictCell>
                <es:enbId>915606</es:enbId>
                <es:cellId>2</es:cellId>
                <es:mcc>404</es:mcc>
                <es:mnc>10</es:mnc>
                <es:mncLength>2</es:mncLength>
                </es:pciConflictCell>
                <es:pciConflictCell>
                <es:enbId>915606</es:enbId>
                <es:cellId>2</es:cellId>
                <es:mcc>404</es:mcc>
                <es:mnc>10</es:mnc>
                <es:mncLength>2</es:mncLength>
                </es:pciConflictCell>
                <es:cellId>1</es:cellId>
                </es:vsDataEUtranCellFDD>
            </xn:attributes>
        </xn:VsDataContainer>

        <xn:VsDataContainer id="LD10118K">
            <xn:attributes>
                <xn:vsDataType>vsDataEUtranCellFDD</xn:vsDataType>
                <xn:vsDataFormatVersion>EricssonSpecificAttributes.16.28</xn:vsDataFormatVersion>
                <es:vsDataEUtranCellFDD>
                <es:userLabel>LTE_LD10118_Sultanpur</es:userLabel>
                <es:frameStartOffset>
                <es:subFrameOffset>0</es:subFrameOffset>
                </es:frameStartOffset>
                <es:administrativeState>1</es:administrativeState>
                <es:earfcndl>1300</es:earfcndl>
                <es:earfcnul>19300</es:earfcnul>
                <es:cellId>1</es:cellId>
                <es:physicalLayerCellIdGroup>138</es:physicalLayerCellIdGroup>
                <es:physicalLayerSubCellId>0</es:physicalLayerSubCellId>
                <es:pciConflict>2</es:pciConflict>
                <es:pciConflict>2</es:pciConflict>
                <es:pciConflict>3</es:pciConflict>
                <es:pciConflict>2</es:pciConflict>
                <es:pciConflict>2</es:pciConflict>
                <es:pciConflictCell>
                <es:enbId>915606</es:enbId>
                <es:cellId>2</es:cellId>
                <es:mcc>404</es:mcc>
                <es:mnc>10</es:mnc>
                <es:mncLength>2</es:mncLength>
                </es:pciConflictCell>
                <es:pciConflictCell>
                <es:enbId>915606</es:enbId>
                <es:cellId>2</es:cellId>
                <es:mcc>404</es:mcc>
                <es:mnc>10</es:mnc>
                <es:mncLength>2</es:mncLength>
                </es:pciConflictCell>
                <es:pciConflictCell>
                <es:enbId>915606</es:enbId>
                <es:cellId>2</es:cellId>
                <es:mcc>404</es:mcc>
                <es:mnc>10</es:mnc>
                <es:mncLength>2</es:mncLength>
                </es:pciConflictCell>
                <es:pciConflictCell>
                <es:enbId>915606</es:enbId>
                <es:cellId>2</es:cellId>
                <es:mcc>404</es:mcc>
                <es:mnc>10</es:mnc>
                <es:mncLength>2</es:mncLength>
                </es:pciConflictCell>
                </es:vsDataEUtranCellFDD>
            </xn:attributes>
</xn:VsDataContainer>

sample&result.zip (2.8 KB)


#38

Let me look at the sample attached to understand what the issue is.


#39

@pavin, the parser has been updated to fix the issue. Thanks for bringing it to our attention.


#40

Hi @Emmanuel,

I want to get the TAC information of ExternalEutrancell.

The MO Structure is:

EUtraNetwork=1,ExternalENodeBFunction=92400-987123,ExternalEUtranCellFDD=cellidentif

but this information does not appear in the vsDataExternalENodeBFunction.csv table


#41

@Lucio, Check vsDataExternalEUtranCellFDD.csv


#42

yes, but where? it doesn’t exist :neutral_face: