Page 1 of 41

[RC] NewsSync 1.5.0

Posted: Sat Jul 26, 2003 5:25 pm
by wineknow
NewsSync for phpBB
- syncronizes messages between usenet and phpBB

- the original newssync topic was getting a little long, so I have started a new one for the 1.5.x series

There has been a lot of changes for version 1.5.x most importantly:
- UI via the phpbb admin section to controlling newssync
- web-based install
- many speed improvements
- addition for UNIX systems if you don't have CRON capabilities
- support for phpnuke and postnuke
- configuration settings now stored in the database

** RC6
- fixed a few minor bugs

** RC5
- fixed a few minor bugs
- see this post for more details

** RC4
- fix the install for postnuke

** RC3
- fixed install bug where phpnuke would not install
- fixed pseudo_cron -- parse error

** RC2
- no code changes from RC1
- if you were using B7 -- I did make a mistake, you do need to tweek the schema (download newssync_1.5.0_beta_upgrades.tar and run appropriate SQL script) -- note same goes for previous 1.5.0Betas.

** RC1
- fixed issue where new posts within phpBB would not always be sent to usenet
- no additional upgrade information from B7 (just copy files to correct locations)
- previous from B7 see upgrade instructions below...

New Download Site: http://www.coolerclimate.com/downloads/ ... 5.0rc6.tar for NewsSync downloads.

Note: given my schedule I currently cannot offer support for NewsSync right now. Ideally, I am looking for someone to take over this project. Please PM me if you are interested.
-k

Re: NewsSync 1.5.0 beta

Posted: Sat Jul 26, 2003 10:49 pm
by kaisuny
wineknow wrote: First Beta Release Version 1.5.0 beta 1
- NOT for use in a production system
- First Beta Release, so expect some trouble with install and upgrade
- read the README file in the tar file

New Download Site: http://www.homepage.mac.com/wineknow for NewsSync downloads.

any comments would be appreciated
-k


First thanks for your effort and keeping promises on its lasting development . :lol:
Through ACP, it returns a successfully installed message. But except the install menu item in the ACP, I do not find other menu items to control NewsSync while there is not documentation about how it works. :cry:

Posted: Sun Jul 27, 2003 12:47 am
by qnx
first, make sure the admin_newssync_install.php is deleted from the admin directory AFTER the install is done.

second, do a "refresh" or "reload" from your browse and you will see more stuff under the Newssync Menu.

BTW, do you have a normal phpbb, or phpnuke/phpbb, or postnuke/PNphpBB2?

Posted: Sun Jul 27, 2003 1:12 am
by kaisuny
[quote="qnx"]first, make sure the admin_newssync_install.php is deleted from the admin directory AFTER the install is done.

second, do a "refresh" or "reload" from your browse and you will see more stuff under the Newssync Menu.

BTW, do you have a normal phpbb, or phpnuke/phpbb, or postnuke/PNphpBB2?[/quote]

Thanks. The menu appears in the ACP now.

handling of date/time

Posted: Mon Jul 28, 2003 2:16 am
by qnx
for a thread, one post has a header:

Date: Fri, 25 Jul 2003 22:44:17 +0200

the other one has a header:
Date: 25 Jul 2003 21:34:55 GMT

if you convert the first to GMT, it would be 25 Jul 2003 20:44:17

somehow newsync thinks the first one is Jul 26, 2003 - 12:44 AM
so the the second got listed before the first post.

Re: handling of date/time

Posted: Mon Jul 28, 2003 3:30 am
by VANuker
qnx wrote: for a thread, one original post has a header:

Date: Fri, 25 Jul 2003 22:44:17 +0200

the other one (reply) has a header:
Date: 25 Jul 2003 21:34:55 GMT

if you convert the first to GMT, it would be 25 Jul 2003 20:44:17

somehow newsync thinks the first one is Jul 26, 2003 - 12:44 AM
so the the reply got listed before the original post.


Does newssync still thread with other than the thread id? If I'm reading your comment correctly 1.5 now uses the date/time along with the subject to thread properly?

As I see it that's still a problem. First, when importing threads from the usenet newssync now inserts them in thread order, which causes problems for those trying to follow a conversation along in the forum. Posts should be added to the end of a thread in a forum, in whatever order they're inserted. The order really doesn't matter. However, when exporting posts from the forum out to the usenet, it should only be done via the thread id of the post one responded to. Anything else can, and does, cause problems in threading in the usenet. It gets to be a real problem when people change the subject and such.

Just my .02

Posted: Mon Jul 28, 2003 5:04 am
by qnx
sorry for the confusion. let me re-phrase the issue. both first and second post are a reply to the original post. since usenet has a tree structure while phpbb has a flat model, should phpbb list them as orignal, first, second or orignal, second, first? That's why we need to compare the time between first and second.

anyhow, I changed line 315 of inc/nntp_funcs.php to

Code: Select all

$GMT_time = $newtime  - ($offset / 100 * 60 * 60);
it now lists correctly.

Posted: Mon Jul 28, 2003 5:22 am
by LaZeR
Any idea what is causing this on each and every Newsync Admin page at the top :?:

Warning: ob_start(): output handler 'ob_gzhandler' cannot be used twice in /home/lazer/public_html/phpbb/admin/newssync_header.php on line 43

Also, when viewing the NewsSync Console page, the "Output" area is stretching the window thus creating a horizontal scroll bar and needing to scroll to the right to see, then again the phpbb copyright gets pushed to the right also. I temporarily solved this by changing the following 2 lines in newsync_session_log.tpl both to just:

Code: Select all

<table width="100%">
WAS:

Code: Select all

<table cellspacing="0" cellpadding="4" border="0" align="left" width="100%">
and

Code: Select all

<table width="100%" cellpadding="4" cellspacing="1" border="0" class="forumline" align="left">
And also again I just realized that my host thought it would be cool to disable SSH access w/o telling me so now I can't set the owner permissions on /newssync thus I had to just CHMOD to 777 and the dir.s within /newssync to get this to work at all. Any way around that yet as I don't like leaving it this way for security reasons?

forum name change

Posted: Mon Jul 28, 2003 6:17 am
by qnx
if I change the name of the forum that is synced with newsgroup, will it cause trouble to newsync? I seem to remember the old 1.0 version has the forum name hardcoded in one of the configs...

Posted: Mon Jul 28, 2003 1:05 pm
by LaZeR
Just another observation - possible problem.

I had a few hangs when initially doing the sync (Setting for initial was set to 100). Later I succeeded after purging out completely both my 2 usenet forums. But when done it seemed to pickup real old dated topics (probably the oldest as in clear back to May and none this month) 8O So I re-purged and set the initial to 0 for both forums-feeds. Hehe, now I'm nearly 1 1/2 hours into it and its definetly retreiving and also grabbing some in July so far but oh boy this could be a problem even if it is a one time deal because I imagine this script will be running the best part of the day to catch up to both groups as one contains 11,000 + and the other like 42,000 and I'm only up to about 4,000 posts in the first forum but I figured the way this is working now, I had to do a total dump to have a shot at getting the latest dated posts from now on :?:

Note: Before I re-purged and left the setting @ 100, I tried several times running again from the admin console but it would never find anything newer than May

Hope that makes some sort of sense :?

Posted: Mon Jul 28, 2003 6:02 pm
by LaZeR
Yet another :?: and possible problem.

Have I missed something or is it no longer possible to run as a chron with this version? Have tried several methods as follows:

Previously I had the "gosync" setup which worked with the old v1 version but now it doesn't seem to fire the script and my chron reports back as follows:
/home/lazer/public_html/phpbb/gosync: line 1: cd: /home/lazer/public_html/phpbb/newssync: No such file or directory
No input file specified.


Chron line example:

0 * * * * /home/lazer/public_html/phpbb/gosync &1>/dev/null &2>/dev/null

contents of gosync

cd /home/lazer/public_html/phpbb/newssync
/usr/bin/php -q newssync.php


Now if I login SSH and:

cd /home/lazer/public_html/phpbb/newssync

then type:

/usr/bin/php -q newssync.php

I now get the following:
<html>
<body>
phpBB : <b>Critical Error</b>
<br /><br />
Could not connect to the database</body>
</html>


Maybe I'm doing something wrong but I can't seem to find any documentation outlaying the changes (specifically cron related) to invoke when trying this new beta with ACP support. :? :?: :cry:

Or is it only possible to run manually now for syncing through the ACP function? (Is the only thing that I can seem to make work at all now) :oops:

Posted: Mon Jul 28, 2003 6:16 pm
by wineknow
check your permissions on the files, the user running the cron job must have access to all the files. Can you run newssync from the NewsSync console in phpbb admin?

LaZeR wrote: Yet another :?: and possible problem.

Have I missed something or is it no longer possible to run as a chron with this version? Have tried several methods as follows:

Previously I had the "gosync" setup which worked with the old v1 version but now it doesn't seem to fire the script and my chron reports back as follows:
/home/lazer/public_html/phpbb/gosync: line 1: cd: /home/lazer/public_html/phpbb/newssync: No such file or directory
No input file specified.


Chron line example:

0 * * * * /home/lazer/public_html/phpbb/gosync &1>/dev/null &2>/dev/null

contents of gosync

cd /home/lazer/public_html/phpbb/newssync
/usr/bin/php -q newssync.php


Now if I login SSH and:

cd /home/lazer/public_html/phpbb/newssync

then type:

/usr/bin/php -q newssync.php

I now get the following:
<html>
<body>
phpBB : <b>Critical Error</b>
<br /><br />
Could not connect to the database</body>
</html>


Maybe I'm doing something wrong but I can't seem to find any documentation outlaying the changes (specifically cron related) to invoke when trying this new beta with ACP support. :? :?: :cry:

Or is it only possible to run manually now for syncing through the ACP function? (Is the only thing that I can seem to make work at all now) :oops:

Re: forum name change

Posted: Mon Jul 28, 2003 6:18 pm
by wineknow
qnx wrote: if I change the name of the forum that is synced with newsgroup, will it cause trouble to newsync? I seem to remember the old 1.0 version has the forum name hardcoded in one of the configs...


should have no impact.
the database now stores the config settings and it no longer syncs to a forum name but a forum id (like the rest of phpbb), so if you rename a forum nothing should break...

now if you change the usenet group that is a different story.

-k

Re: handling of date/time

Posted: Mon Jul 28, 2003 6:24 pm
by wineknow
qnx wrote: for a thread, one post has a header:

Date: Fri, 25 Jul 2003 22:44:17 +0200

the other one has a header:
Date: 25 Jul 2003 21:34:55 GMT

if you convert the first to GMT, it would be 25 Jul 2003 20:44:17

somehow newsync thinks the first one is Jul 26, 2003 - 12:44 AM
so the the second got listed before the first post.


you may have fixed a big bug for me -- I have looked at this code way to much to have missed this....sorry folks.

just to make sure I get it right
Fri, 25 Jul 2003 22:44:17 +0200
== 25 Jul 2003 20:44:17 GMT
and NOT 25 Jul 2003 24:44:17 GMT

right now I am +2h and should be -2h...


-k

Posted: Mon Jul 28, 2003 7:02 pm
by LaZeR
wineknow wrote: check your permissions on the files, the user running the cron job must have access to all the files. Can you run newssync from the NewsSync console in phpbb admin?


Well this is the part I'm probably not clear on because I had to end up CHMOD via FTP the /newssync DIR to 777 along with /newssync/data

But I had run the following in SSH session:

chown -R lazer newssync (But still had to 777 with CHMOD) :?

I did put an .htaccess file with: deny from all in /newssync also.

Runs fine in admin but the former way I was able to run the cron with gosync fails now. What am I missing?