How do Process 3rd Party Data?

Relax and enjoy The Lounge. For all Non-Joomla! topics or ones that don't fit anywhere else. Normal forum rules apply.
MyFirstPage
Joomla! Enthusiast
Joomla! Enthusiast
Posts: 131
Joined: Thu Feb 25, 2010 4:37 pm

How do Process 3rd Party Data?

Postby MyFirstPage » Sun Jan 22, 2017 12:05 am

My Gov Publish OpenData who can used by everyone. Some of them get Updated Automaticly every 5, 10, 20, 30, 60,... Minutes and other every Day. :eek:
The 3 most used Files: CSV (1306), shp (558), JSON (437)

so 99% of the Data need to proceed and make it smooth and cosy to fit together. The Main Problem every City, Districts, federal State,... desired what the Publish or not. That mean I have to deal with different formats for the same Topic.

Next Problem is other Data (like Postcode Data for example) must be cominded.

Any idea how to make? It must run 24/7 w/o any user input.
Thanks

User avatar
sone12
Joomla! Champion
Joomla! Champion
Posts: 5435
Joined: Wed Oct 11, 2006 1:12 pm
Location: Kristiansand, Norway
Contact:

Re: How do Process 3rd Party Data?

Postby sone12 » Thu Aug 17, 2017 6:33 am

If you want to collect and use data from multiple services, you will always see that the services have their own way to do things, and you will always need to do a lot of coding to format the data to fit your own application.

What you can do, is to ask each service the try follow some kind of standards and so on. Maybe you can show them your needs, and what they can do better to not make trouble for you.
Kristian P. Granrud - Global Moderator

MyFirstPage
Joomla! Enthusiast
Joomla! Enthusiast
Posts: 131
Joined: Thu Feb 25, 2010 4:37 pm

Re: How do Process 3rd Party Data?

Postby MyFirstPage » Thu Aug 17, 2017 8:46 am

Nope its simple everyone can do whit the Data what but he have to find a solution to use it.
The are "just" CSV Files but the need to processed. The only Solution I would know is when I process them on a Virtual Linux at Home. (I dont know how or if it will work. I never use a Linux System before.)
But how to get the Data into the Homepage?! I would say the best thing is get the Data into a SQL Database.
But how to get the Data into the SQL Database?

User avatar
sone12
Joomla! Champion
Joomla! Champion
Posts: 5435
Joined: Wed Oct 11, 2006 1:12 pm
Location: Kristiansand, Norway
Contact:

Re: How do Process 3rd Party Data?

Postby sone12 » Thu Aug 17, 2017 9:11 am

You can read and write csv content with e.g php code. If you want to put the content into a database is up you you, but you dont need to. A good startingpoint is fgetcsv() to get/read the content, and then you can do what you want, print it out og save to database or both. There is also complete scripts shared and free for you to use, on git and in composer.
Kristian P. Granrud - Global Moderator

MyFirstPage
Joomla! Enthusiast
Joomla! Enthusiast
Posts: 131
Joined: Thu Feb 25, 2010 4:37 pm

Re: How do Process 3rd Party Data?

Postby MyFirstPage » Thu Aug 17, 2017 9:15 am

Cool but how does it look like in the end?
Something like this:
step-2.jpg

In the Joomla Group from Germany I have to argue around who I not want to show it like the picture above.
It have to be integrated smoothly what no one know which System is behind. And I have to add a Copyright.
You do not have the required permissions to view the files attached to this post.

User avatar
sone12
Joomla! Champion
Joomla! Champion
Posts: 5435
Joined: Wed Oct 11, 2006 1:12 pm
Location: Kristiansand, Norway
Contact:

Re: How do Process 3rd Party Data?

Postby sone12 » Thu Aug 17, 2017 9:20 am

Sure you can output the content from a csv like that, in a table. But you need to add some javascript to get those filters on top.
Kristian P. Granrud - Global Moderator

MyFirstPage
Joomla! Enthusiast
Joomla! Enthusiast
Posts: 131
Joined: Thu Feb 25, 2010 4:37 pm

Re: How do Process 3rd Party Data?

Postby MyFirstPage » Thu Aug 17, 2017 9:25 am

nnnnnnnnnnnoooooooooooo I dont want them in a Table!!!!
I will make my Subpage like the Wikipedia Page: https://en.wikipedia.org/wiki/Linz
with Weather and other Realtime Data.

User avatar
sone12
Joomla! Champion
Joomla! Champion
Posts: 5435
Joined: Wed Oct 11, 2006 1:12 pm
Location: Kristiansand, Norway
Contact:

Re: How do Process 3rd Party Data?

Postby sone12 » Thu Aug 17, 2017 9:49 am

MyFirstPage wrote:nnnnnnnnnnnoooooooooooo I dont want them in a Table!!!!
I will make my Subpage like the Wikipedia Page: https://en.wikipedia.org/wiki/Linz
with Weather and other Realtime Data.
You uploaded a picture of a table, and I answered you that you can get it like that if you want ;)

Basicly, when you have loaded the content from csv into php, you probably have it as an object or an array. You can do what ever you want with it. You can print it as a table, create joomla articles of it, you can manipulate it before printing, save it to database, print it as a list, send it as email or something else. It is all about your php skillz and fantasy.

Of course, you can also connect and send needed data to a weather station rest service, and get information about the weather and such.
Kristian P. Granrud - Global Moderator

MyFirstPage
Joomla! Enthusiast
Joomla! Enthusiast
Posts: 131
Joined: Thu Feb 25, 2010 4:37 pm

Re: How do Process 3rd Party Data?

Postby MyFirstPage » Thu Aug 17, 2017 10:07 am

Sure I know my English skills are not the best.

It is all about your php skillz and fantasy.
Both are 0.
Is there any Information how to do it?

gws
Joomla! Hero
Joomla! Hero
Posts: 2744
Joined: Tue Aug 23, 2005 1:56 pm
Location: Kent / Sussex / Surrey border UK
Contact:

Re: How do Process 3rd Party Data?

Postby gws » Mon Sep 11, 2017 1:10 pm


MyFirstPage
Joomla! Enthusiast
Joomla! Enthusiast
Posts: 131
Joined: Thu Feb 25, 2010 4:37 pm

Re: How do Process 3rd Party Data?

Postby MyFirstPage » Wed Sep 20, 2017 2:44 pm

thx I dont understand it that much buy i will save it for later.


Return to “The Lounge”

Who is online

Users browsing this forum: No registered users and 9 guests