Channels
#tornadofx
- j
jchildress
02/07/2017, 6:52 PM@edvin It seems nice to me and your explanation is clear. I confess I haven't tried this new functionality yet. I have seen you guys talking about it here, but I have been swamped with some exciting ETL work lately 😞 - c
carlw
02/07/2017, 6:55 PM@jchildress what tool do you use for ETL? - p
permalmberg
02/07/2017, 6:55 PMwhat is ETL? 🙂 - c
carlw
02/07/2017, 6:55 PMextract/transform/load - j
jchildress
02/07/2017, 6:55 PMthis job was manual... downloaded a bunch of zip/csv files... used bcp to load them in sql server... then started on my analysis/aggregation... - j
jchildress
02/07/2017, 6:56 PMpage was clean... use jsoup to get all links and then download... there wasn't really any T... - j
jchildress
02/07/2017, 6:56 PMextract transform load... - c
carlw
02/07/2017, 6:56 PMi see - p
permalmberg
02/07/2017, 6:56 PMoh, data processing - p
permalmberg
02/07/2017, 6:57 PM🙂 - j
jchildress
02/07/2017, 6:57 PMwe are in the early parts of this "project"... - p
permalmberg
02/07/2017, 6:58 PMgotta love IT - there are at leat 50 different terminologies for each thing we do... 😛 - j
jchildress
02/07/2017, 6:58 PMindeed... - p
permalmberg
02/07/2017, 6:59 PMWe do ETL daily...just didn't know there was a term for it 🙂 - c
carlw
02/07/2017, 7:00 PM@permalmberg pretty established in data warehousing...sometimes you see ELT which is a performance improvement that transforms within the database (instead of shuttling data all around various process via IO or the network) - p
permalmberg
02/07/2017, 7:01 PMyeah, I'd call it normalizing data, but then I'm no DBA - c
carlw
02/07/2017, 7:03 PMit's the whole activity...batch processing involving extraction (pulling data from an external source), loading (CSV -> RDBMS) , and transforming (flat denormalized structure to RDBMS) - c
carlw
02/07/2017, 7:03 PMI was curious to see if john was using a tool like Informatica or DataStage which is usually involved when there's a lot of this type of work - p
permalmberg
02/07/2017, 7:03 PMyup. - j
jchildress
02/07/2017, 7:05 PMno, i would say for most of our production work we have a pretty extensive internal library... over 1,000 "scrapers"... spring batch "manages" them... - j
jchildress
02/07/2017, 7:05 PMmost of the data is pretty clean... xml or csv - j
jchildress
02/07/2017, 7:05 PMsome is a little messy.... - r
Ruckus
02/07/2017, 7:05 PMSounds pretty good to me, but I haven't messed with it yet. - c
carlw
02/07/2017, 7:21 PM@edvin i recalled my last message...enterprise architect has 2 nav controls. the one most similar to yours is over the drawers like in intellij - p
permalmberg
02/07/2017, 7:21 PMOh, another one who uses EA 🙂 - c
carlw
02/07/2017, 7:22 PM@User uploaded a file:and commented: Similar nav control placementPasted image at 2017-02-07, 2:21 PM▾
- p
permalmberg
02/07/2017, 7:22 PMFor some reason it has gotten bad rep... I like it. It does what I want it to 🙂 - c
carlw
02/07/2017, 7:22 PMSparxSystems specifically or UML in general? - p
permalmberg
02/07/2017, 7:22 PMSparx - p
permalmberg
02/07/2017, 7:24 PMThough I'm fighting against the current to introduce UML at work. But each time I can answer "how does it work?" by whipping out the UML, I get one step closer 😄