Tweepository - Revision history

Af05v@ecs.soton.ac.uk: /* Exporting to a Web Observatory */

2015-12-18T12:22:48Z

Exporting to a Web Observatory

← Older revision		Revision as of 12:22, 18 December 2015
Line 210:		Line 210:
	** More that one tweetstream can be inserted into a collection		** More that one tweetstream can be inserted into a collection
	** The URL of the tweepository will be prepeneded onto the collection ID. If 'foo' is the collection name, then on the Web Observatory, the collection will be (e.g.) 'http://www.mytweepository.com/foo'		** The URL of the tweepository will be prepeneded onto the collection ID. If 'foo' is the collection name, then on the Web Observatory, the collection will be (e.g.) 'http://www.mytweepository.com/foo'


			=== Nightly Export ===

			The web_observatory_push.pl script needs to be added to the crontab to run nightly. It should be run after the update_tweetstream_abstracts.pl script. Note that it is blocked by update_tweetstream_abstracts, so will only start doing actual work when that script has terminated.

	== Troubleshooting ==		== Troubleshooting ==

Af05v@ecs.soton.ac.uk: /* Configuring a Web Observatory */

2015-12-18T12:19:51Z

Configuring a Web Observatory

← Older revision		Revision as of 12:19, 18 December 2015
Line 196:		Line 196:

	perl -I/<eprints_root>/perl_lib /<eprints_root>/archives/<repositoryid>/bin/tweepository/web_observatory_test.pl <repositoryid> <webobservatoryid>		perl -I/<eprints_root>/perl_lib /<eprints_root>/archives/<repositoryid>/bin/tweepository/web_observatory_test.pl <repositoryid> <webobservatoryid>

			==== authorised_users ====

			A tweetstream is allowed to be pushed to a Web Observatory if its the person that created the tweetstream is in the list of authorised users.

	=== Configuring a TweetStream ===		=== Configuring a TweetStream ===

Af05v@ecs.soton.ac.uk: /* Configuring a Web Observatory */

2015-12-18T12:17:44Z

Configuring a Web Observatory

← Older revision		Revision as of 12:17, 18 December 2015
Line 173:		Line 173:
	An example web observatory configuration is visible in the z_tweepository_cfg.pl file. This should be copied to a new .pl file and an entry created for each Web Observatory database that is configured:		An example web observatory configuration is visible in the z_tweepository_cfg.pl file. This should be copied to a new .pl file and an entry created for each Web Observatory database that is configured:

	<source lang="perl'>		<source lang="perl">
	$c->{web_observatories} =		$c->{web_observatories} =
	{		{
Line 196:		Line 196:

	perl -I/<eprints_root>/perl_lib /<eprints_root>/archives/<repositoryid>/bin/tweepository/web_observatory_test.pl <repositoryid> <webobservatoryid>		perl -I/<eprints_root>/perl_lib /<eprints_root>/archives/<repositoryid>/bin/tweepository/web_observatory_test.pl <repositoryid> <webobservatoryid>


	=== Configuring a TweetStream ===		=== Configuring a TweetStream ===

Af05v@ecs.soton.ac.uk: /* Configuring a TweetStream */

2015-12-18T12:17:05Z

Configuring a TweetStream

← Older revision		Revision as of 12:17, 18 December 2015
Line 205:		Line 205:
	* The ID of the web observatory -- In the above example, it would be 'webobservatory1'		* The ID of the web observatory -- In the above example, it would be 'webobservatory1'
	* Web Observatory Collection -- the MongoDB collection that this will be inserted into. Note that:		* Web Observatory Collection -- the MongoDB collection that this will be inserted into. Note that:
			** More that one tweetstream can be inserted into a collection
			** The URL of the tweepository will be prepeneded onto the collection ID. If 'foo' is the collection name, then on the Web Observatory, the collection will be (e.g.) 'http://www.mytweepository.com/foo'

	== Troubleshooting ==		== Troubleshooting ==

Af05v@ecs.soton.ac.uk: /* Exporting to a Web Observatory */

2015-12-18T12:15:02Z

Exporting to a Web Observatory

← Older revision		Revision as of 12:15, 18 December 2015
Line 166:		Line 166:

	== Exporting to a Web Observatory ==		== Exporting to a Web Observatory ==

			Tweet data can be exported nightly to a Web Observatory. This is accomplished by inserting tweet records into an external MongoDB database that has been set up by the administrators of the Web Observatory for this purpose.

			=== Configuring a Web Observatory ===

			An example web observatory configuration is visible in the z_tweepository_cfg.pl file. This should be copied to a new .pl file and an entry created for each Web Observatory database that is configured:

			<source lang="perl'>
			$c->{web_observatories} =
			{
			'websobservatory1' =>
			{
			type => 'mongodb',
			server => 'foo.bar.uk',
			port => 1234,
			db_name => 'foo',
			username => 'username',
			password => 'password',
			authorised_users =>
			[
			'username1',
			'username2'
			]
			}
			};
			</source>

			To test, use the web_observatory_test.pl script:

			perl -I/<eprints_root>/perl_lib /<eprints_root>/archives/<repositoryid>/bin/tweepository/web_observatory_test.pl <repositoryid> <webobservatoryid>


			=== Configuring a TweetStream ===

			When editing or creating a tweetstream, fill in the Web Observatory Export part of the workflow:

			* Export To Web Observatory -- set to 'Yes'
			* The ID of the web observatory -- In the above example, it would be 'webobservatory1'
			* Web Observatory Collection -- the MongoDB collection that this will be inserted into. Note that:

	== Troubleshooting ==		== Troubleshooting ==

Af05v@ecs.soton.ac.uk at 11:54, 18 December 2015

2015-12-18T11:54:31Z

← Older revision		Revision as of 11:54, 18 December 2015
Line 164:		Line 164:

	These three roles have been assigned to repository administrators, editors and users respectively. This can be changed by modifying this part of the config.		These three roles have been assigned to repository administrators, editors and users respectively. This can be changed by modifying this part of the config.

			== Exporting to a Web Observatory ==

	== Troubleshooting ==		== Troubleshooting ==

Af05v@ecs.soton.ac.uk: /* Add Twitter API Keys */

2014-06-12T09:35:54Z

Add Twitter API Keys

← Older revision		Revision as of 09:35, 12 June 2014
Line 34:		Line 34:

	<pre>		<pre>
			$c->{twitter_oauth}->{consumer_key} = 'xxxxxxxxxxxx';
			$c->{twitter_oauth}->{consumer_secret} = 'xxxxxxxxxxxxxxxxxxxxxxxx';
			$c->{twitter_oauth}->{access_token} = 'xxxxxxxxxxxx-xxxxxxxxxxxxxxxxxxxxxxxx';
			$c->{twitter_oauth}->{access_token_secret} = 'xxxxxxxxxxxxxxxxxxxxxxxx';
	</pre>		</pre>

Af05v@ecs.soton.ac.uk at 09:22, 12 June 2014

2014-06-12T09:22:36Z

← Older revision		Revision as of 09:22, 12 June 2014
Line 28:		Line 28:

	Install through the EPrints Bazaar		Install through the EPrints Bazaar

			== Add Twitter API Keys ==

			Create a file in your repository's cfg.d directory for your twitter API keys (e.g. called z_tweepository_oath.pl). The content should look like this:

			<pre>


			</pre>

	== Setting Up Cron Jobs ==		== Setting Up Cron Jobs ==

Af05v@ecs.soton.ac.uk: /* Installation Prerequisites */

2013-01-02T14:24:28Z

Installation Prerequisites

← Older revision		Revision as of 14:24, 2 January 2013
Line 8:		Line 8:

	<pre>		<pre>
			Archive::Zip
			Archive::Zip::MemberRead
	Data::Dumper		Data::Dumper
	Date::Calc		Date::Calc
	Date::Parse		Date::Parse
	Encode		Encode
			File::Copy
			File::Path
	HTML::Entities		HTML::Entities
	JSON		JSON
	LWP::UserAgent		LWP::UserAgent
	Number::Bytes::Human		Number::Bytes::Human
	Storable		Storable
	URI		URI
	URI::Find		URI::Find

Af05v@ecs.soton.ac.uk: /* Setting Up Cron Jobs */

2012-12-08T22:38:14Z

Setting Up Cron Jobs

← Older revision		Revision as of 22:38, 8 December 2012
Line 27:		Line 27:
	== Setting Up Cron Jobs ==		== Setting Up Cron Jobs ==

	There are three processes that need to be regularly run. Due to the heavyweight nature of these tasks, they should be put into the crontab rather than being handled by the EPrints Indexer. However, they have been created as event plugins for future inclusion in the indexer. Scripts have been created in the ~~plugin directory.~~		There are three processes that need to be regularly run. Due to the heavyweight nature of these tasks, they should be put into the crontab rather than being handled by the EPrints Indexer. However, they have been created as event plugins for future inclusion in the indexer. Scripts have been created to wrap the plugins

	Running these scripts will result in log files and a cache being created in the repository's 'var' directory. These should be checked if there are any issues with harvesting.		Running these scripts will result in log files and a cache being created in the repository's 'var' directory. These should be checked if there are any issues with harvesting.