How to make LWP and HTML::TableExtract spitting out CSV with Text::CSV?

Since you have entered a complete script, I'll assume you want critique of the whole thing.

Since you have entered a complete script, I'll assume you want critique of the whole thing. #! /usr/bin/perl use strict; use warnings; use HTML::TableExtract; use LWP::Simple; use Cwd; use POSIX qw(strftime); my $te = HTML::TableExtract->new; Since you only use $te in one block, why are you declaring and initializing it in this outer scope?

The same question applies to most of your variables -- try to declare them in the innermost scope possible. My $total_records = 0; my $suchbegriffe = "e"; my $treffer = 50; In general, english variable names will enable you to collaborate with far more people than german names. I understand german, so I understand the intent of your code, but most of SO doesn't.My $range = 0; my $url_to_process = "http://192.68.214.70/km/asps/schulsuche.

Asp? Q="; my $processdir = "processing"; my $counter = 50; my $displaydate = ""; my $percent = 0; &workDir(); Don't use & to call subs. Just call them with workDir;.

It hasn't been necessary to use & since 1994, and it can lead to a nasty gotcha because &callMySub; is a special case which doesn't do what you might think, while callMySub; does the Right Thing. Chdir $processdir; &processURL(); print "\nPress to continue\n"; ; $displaydate = strftime('%Y%m%d%H%M%S', localtime); open OUTFILE, ">webdata_for_$suchbegriffe\_$displaydate. Txt"; Generally lexical filehandles are preferred these days: open my $outfile, ">file"; Also, you should check for errors from open or use autodie; to make open die on failure.

&processData(); close OUTFILE; print "Finished processing $total_records records...\n"; print "Processed data saved to $ENV{HOME}/$processdir/webdata_for_$suchbegriffe\_$displaydate. Txt\n"; unlink 'processing. Html'; die "\n"; sub processURL() { print "\nProcessing $url_to_process$suchbegriffe&a=$treffer&s=$range\n"; getstore("$url_to_process$suchbegriffe&a=$treffer&s=$range", 'tempfile.

Html') or die 'Unable to get page'; while( ) { open( FH, "$_" ) or die; while( ) { if( $_ =~ /^. *?(Treffer )(d+)( - )(d+)( w+ w+ )(d+). */ ) { $total_records = $6; print "Total records to process is $total_records\n"; } } close FH; } unlink 'tempfile.

Html'; } sub processData() { while ( $range parse_file('processing. Html'); my ($table) = $te->tables; for my $row ( $table->rows ) { cleanup(@$row); print OUTFILE "@$row\n"; This is the line to change if you want to put commas in separating your data. Look at the join function, it can do what you want.

} $| = 1; print "Processed records $range to $counter"; print "\r"; $counter = $counter + 50; $range = $range + 50; $te = HTML::TableExtract->new; } It's very strange to initialize $te at the end of the loop instead of the beginning.It's much more idiomatic to declare and initialize $te at the top of the loop. } sub cleanup() { for ( @_ ) { s/s+/ /g; Did you mean s/\s+/ /g;? } } sub workDir() { # Use home directory to process data chdir or die "$!"; if (! -d $processdir ) { mkdir ("$ENV{HOME}/$processdir", 0755) or die "Cannot make directory $processdir: $!

"; } } I haven't commented on your second script; perhaps you should ask it as a separate question.

Philip - many many thanks for the hints. Great to see such - in-depth-going critique! I will walk trough and have a look.It is great to see you helping me to code better.

Many many thanks. It is a great pleasure to be here! – zero Feb 20 at 12:33 @ Philip: regarding your ideas concerning the second script: I want to migrate the both.

Can you (or someone else) give me some tips how to do both scripts in one... As being the novice - I need a helping hand here.. - all hints and tips greatly appreciated. Zero – zero Feb 20 at 12:36 1 @zero If you have found this answer useful, please upvote it. That's the stackoverflow way of saying "thanks".

– Philip Potter Feb 20 at 12:36 hi Philip. This answer was more than useful. Can you give me some concrete hints to migrate the both scripts.

Note - friends helped me with the scripts. So, you see, my Perl-knowlgedge is not so elaborated that I am able to do the migration into one on my own! Any and all help would be great!

-greetings zero – zero Feb 20 at 12:50 3 “Generally lexical filehandles are preferred these days: open my $outfile, ">file";”: also use three-argument open: open my $outfile, '>', $file;. – mscha Feb 20 at 14:00.

I cant really gove you an answer,but what I can give you is a way to a solution, that is you have to find the anglde that you relate to or peaks your interest. A good paper is one that people get drawn into because it reaches them ln some way.As for me WW11 to me, I think of the holocaust and the effect it had on the survivors, their families and those who stood by and did nothing until it was too late.