This document was generated from CDN thread
Created by: Koushal Pandit on 04-02-2013 05:53:44 AM
So I am trying to populate the database with posts from the feed, the function requires xml data from the feed. So I use the following for the data :
http://<server-name>/ccp-webapp/ccp/campaign/CampaignName/results
but it is missing out certain posts/comments. Although all the post data is available in Social Miner but the results page doesn't show them. Is it the correct link for the xml data of the feed? Any reason why it's not showing all the data?
Subject: RE: New Message from Koushal Pandit in Cisco SocialMiner - Technical Discus
Replied by: Tetyana Shmukler on 04-02-2013 11:58:23 AM
Hi,
Please clarify if you not getting the results you wanted using SocialMiner REST API campaign results call? Are you refereeing to some specific time frame that the results are missing for?
Also it is not clear if you also not getting the expected results in SocialMIner UI HOME page for this specific campaign?
Please clarify.
In any case, there are detail explanations in User Guide on how to use campaign results api with an additional parameters and what happens if those are not provided (what kind of defaults are used instead). Please refer to this document. I pasted some details below , but refer to the document for the complete reference:
==============================================================================
•timestamp: Integer. Display results older results than this timestamp. Defaults to the time of request if
not provided. If startIndex is not specified then timestamp assumes startIndex = 0.
• resultsPerPage: Integer. The maximum number of results to be returned. Default is 50 and maximum
is 200.
• startIndex: Integer. The number of results to skip based on the timestamp. Used for pagination. Assuming
resultsPerPage is set at the default of 50, then you could create a “page 2” link by using the timestamp
provided in the href of the feed/link rel="self" and a startIndex of 50. Page 3 would use the same
timestamp and a startIndex of 100, and so on.
Note
*If timestamp is provided and startIndex is not provided, then the results are displayed up to the
“resultsPerPage” with a creation date older than “timestamp”, starting at index 0.
*If timestamp is not provided and startIndex is provided, then the results are displayed up to the
“resultsPerPage” with a creation date older than “now” starting at startIndex.
For SocialMIner UI, please look if there may be an additional filter set for the campaign or there is a tag filtering going on (you can check it in Settings). That can narrow down the original results set.
You can also look at the ccpapi logs to see what query is used , when you select the campaign on HOME page. Please also note that UI will also filter based on the SocialContact state chosen (unread, handled, etc.)
From the logs you can see how query looks like and try the corresponding REST api call.
Please find the example of the logs , I am referring to, below .
If your questions still remain un-answered, please provide us with the exact api call, logs and the explanations on what data are missing , etc.
Thank you
Tanya
>>>>>>>>>>>>>>> snippet from ccpapi log :
0000015537: 10.86.141.221: Feb 04 2013 10:23:28.639 -0500: %CCBU_INFRASTRUCTURE-6-REQUEST_START: %[ip.orig=10.86.141.221][method_name=GET][parameter_name={ time=[1359991408593], filterStatus= }][port.orig=46355][resource_name=/campaign/Test_Camp/results]: Request start
0000000637: 10.86.141.221: Feb 04 2013 10:23:28.643 -0500: %CCBU_____CCP-DS-API-6-CCP_DS_API_MESSAGE: SolandraDS.getByCampaign query: campaignStatus_106162_s:IN AND de.type:SocialContact AND sc.socialContactStatus
unread) [from:null, limit:51]
0000000638: 10.86.141.221: Feb 04 2013 10:23:28.658 -0500: %CCBU_____CCP-DS-API-6-CCP_DS_API_MESSAGE: Stored new cache entry com.cisco.ccbu.ccp.ds.cassandra.api.CampaignQueryCache$CacheEntry@42ec93[campaignId=106162,params=com.cisco.ccbu.ccp.ds.beans.CampaignQueryParams@1181a7d[limit=51,fromIndex=<null>,campaignStatus=IN,baseTimestamp=<null>,timestamp=<null>,filterStatus=,filterTag=<null>,socialContact=<null>,socialContactId=<null>,solrSearchQuery=<null>,includeContactWithContactId=false],lastPublished=Wed Oct 31 20:11:32 EDT 2012,firstPublished=Mon Aug 06 04:34:27 EDT 2012]
0000004023: 10.86.141.221: Feb 04 2013 10:23:28.770 -0500: %CCBU_________CCPAPI-6-CAMPAIGN_RESULTS_FOUND: %[query params=com.cisco.ccbu.ccp.ds.beans.CampaignQueryParams@1cb70bf[limit=51,fromIndex=<null>,campaignStatus=IN,baseTimestamp=<null>,timestamp=<null>,filterStatus=,filterTag=<null>,socialContact=<null>,socialContactId=<null>,solrSearchQuery=<null>,includeContactWithContactId=false]]: Found and returning campaign results for query
0000015538: 10.86.141.221: Feb 04 2013 10:23:28.795 -0500: %CCBU_INFRASTRUCTURE-6-REQUEST_END: %[PARAM_TIME_ELAPSED=155]: Request complete
0000015539: 10.86.141.221: Feb 04 2013 10:23:29.111 -0500: %CCBU_INFRASTRUCTURE-6-REQUEST_START: %[ip.orig=10.86.141.221][method_name=GET][parameter_name={ time=[1359991409065], filterStatus=, postId=[B94F54821000013A000070360A568DDD] }][port.orig=46355][resource_name=/campaign/Test_Camp/count]: Request start
0000015540: 10.86.141.221: Feb 04 2013 10:23:29.123 -0500: %CCBU_INFRASTRUCTURE-6-REQUEST_END: %[PARAM_TIME_ELAPSED=11]: Request complete
From: Cisco Developer Community Forums [mailto:cdicuser@developer.cisco.com]
Sent: Monday, February 04, 2013 6:54 AM
To: cdicuser@developer.cisco.com
Subject: New Message from Koushal Pandit in Cisco SocialMiner - Technical Discussion: Xml data Result
Koushal Pandit has created a new message in the forum "Technical Discussion": -------------------------------------------------------------- So I am trying to populate the database with posts from the feed, the function requires xml data from the feed. So I use the following for the data :
http://<server-name>/ccp-webapp/ccp/campaign/CampaignName/results<http://%3cserver-name%3e/ccp-webapp/ccp/campaign/CampaignName/results>
but it is missing out certain posts/comments. Although all the post data is available in Social Miner but the results page doesn't show them. Is it the correct link for the xml data of the feed? Any reason why it's not showing all the data?
--
To respond to this post, please click the following link: http://developer.cisco.com/web/socialminer/forums/-/message_boards/view_message/11514985 or simply reply to this email.
Subject: RE: New Message from Koushal Pandit in Cisco SocialMiner - Technical Discus
Replied by: Koushal Pandit on 07-02-2013 12:48:09 AM
I am not getting the results I wanted using SocialMiner REST API campaign results call. I am not refereeing to a specific time frame that the results are missing for. I am getting the expected results in SocialMIner UI HOME page for this specific campaign. There only some posts that are missing not all.
I even checked cccapi for the correct API call and still the result REST API is not displaying some of the posts even though the SocialMiner UI home is showing those posts when I search them.
Subject: RE: New Message from Koushal Pandit in Cisco SocialMiner - Technical Discus
Replied by: Tetyana Shmukler on 07-02-2013 12:16:23 PM
Hi,
Thank you for getting back to us.
It is hard to troubleshoot based on the provided information. It would help, if you can provide us with the details on what feed you use (its type, url, pooling interval),, as well as the actual API call (the http request), so we can try to reproduce the behavior you see in house.
With the information above, it would be great to have an example of at least one SocialContact (post) that is missing from the result of your API call.
Please also specify what version of SocialMiner you use.
Thank you
Tanya
From: Cisco Developer Community Forums [mailto:cdicuser@developer.cisco.com]
Sent: Thursday, February 07, 2013 1:48 AM
To: cdicuser@developer.cisco.com
Subject: New Message from Koushal Pandit in Cisco SocialMiner - Technical Discussion: RE: New Message from Koushal Pandit in Cisco SocialMiner - Technical Discus
Koushal Pandit has created a new message in the forum "Technical Discussion": -------------------------------------------------------------- I am not getting the results I wanted using SocialMiner REST API campaign results call. I am not refereeing to a specific time frame that the results are missing for. I am getting the expected results in SocialMIner UI HOME page for this specific campaign. There only some posts that are missing not all.
I even checked cccapi for the correct API call and still the result REST API is not displaying some of the posts even though the SocialMiner UI home is showing those posts when I search them.
--
To respond to this post, please click the following link: http://developer.cisco.com/web/socialminer/forums/-/message_boards/view_message/11649968 or simply reply to this email.
Subject: RE: New Message from Koushal Pandit in Cisco SocialMiner - Technical Discus
Replied by: Koushal Pandit on 08-02-2013 12:51:27 AM
Thank you for your reply. So the we are basically testing out SocialMiner and the link we used was :
https://www.facebook.com/PUBsg with polling interval of 5 mins
the API we called was http://<server-name>/ccp-webapp/ccp/campaign/PUB_SG/results?resultsPerPage=200
One example is the first post by others (on fb page) :
"Next Depress
RC boats at Bedok Reservoir Floating Platform! I like!"
This post comes in SocialMiner UI and in the result API page too but the comments on this post only come in the SocialMiner UI and not the result API.
Another post was "UPDATE: Pipe repair works in progress at Upper Paya Lebar Rd (opp AUPE Building twds Bartley Rd). 1 out of 4 lanes closed. Works expected to be completed by this afternoon. Issued 7:38am"
The comments on this post do not show up in result page. There are many more examples as you can see. This error did not occur in Social Miner 8.5. The social miner 9.0 is a fresh installation and not update of social miner 8.5. We thought this could be an issue regarding a user's privacy level on facebook but there were several posts with same user whose comments were visible in some posts and not visible in other posts on the API result page.
Another anomaly that we noticed was that comments that are made via mobile do not appear either.
Thank you again for all the help.
Created by: Koushal Pandit on 04-02-2013 05:53:44 AM
So I am trying to populate the database with posts from the feed, the function requires xml data from the feed. So I use the following for the data :
http://<server-name>/ccp-webapp/ccp/campaign/CampaignName/results
but it is missing out certain posts/comments. Although all the post data is available in Social Miner but the results page doesn't show them. Is it the correct link for the xml data of the feed? Any reason why it's not showing all the data?
Subject: RE: New Message from Koushal Pandit in Cisco SocialMiner - Technical Discus
Replied by: Tetyana Shmukler on 04-02-2013 11:58:23 AM
Hi,
Please clarify if you not getting the results you wanted using SocialMiner REST API campaign results call? Are you refereeing to some specific time frame that the results are missing for?
Also it is not clear if you also not getting the expected results in SocialMIner UI HOME page for this specific campaign?
Please clarify.
In any case, there are detail explanations in User Guide on how to use campaign results api with an additional parameters and what happens if those are not provided (what kind of defaults are used instead). Please refer to this document. I pasted some details below , but refer to the document for the complete reference:
==============================================================================
•timestamp: Integer. Display results older results than this timestamp. Defaults to the time of request if
not provided. If startIndex is not specified then timestamp assumes startIndex = 0.
• resultsPerPage: Integer. The maximum number of results to be returned. Default is 50 and maximum
is 200.
• startIndex: Integer. The number of results to skip based on the timestamp. Used for pagination. Assuming
resultsPerPage is set at the default of 50, then you could create a “page 2” link by using the timestamp
provided in the href of the feed/link rel="self" and a startIndex of 50. Page 3 would use the same
timestamp and a startIndex of 100, and so on.
Note
*If timestamp is provided and startIndex is not provided, then the results are displayed up to the
“resultsPerPage” with a creation date older than “timestamp”, starting at index 0.
*If timestamp is not provided and startIndex is provided, then the results are displayed up to the
“resultsPerPage” with a creation date older than “now” starting at startIndex.
For SocialMIner UI, please look if there may be an additional filter set for the campaign or there is a tag filtering going on (you can check it in Settings). That can narrow down the original results set.
You can also look at the ccpapi logs to see what query is used , when you select the campaign on HOME page. Please also note that UI will also filter based on the SocialContact state chosen (unread, handled, etc.)
From the logs you can see how query looks like and try the corresponding REST api call.
Please find the example of the logs , I am referring to, below .
If your questions still remain un-answered, please provide us with the exact api call, logs and the explanations on what data are missing , etc.
Thank you
Tanya
>>>>>>>>>>>>>>> snippet from ccpapi log :
0000015537: 10.86.141.221: Feb 04 2013 10:23:28.639 -0500: %CCBU_INFRASTRUCTURE-6-REQUEST_START: %[ip.orig=10.86.141.221][method_name=GET][parameter_name={ time=[1359991408593], filterStatus= }][port.orig=46355][resource_name=/campaign/Test_Camp/results]: Request start
0000000637: 10.86.141.221: Feb 04 2013 10:23:28.643 -0500: %CCBU_____CCP-DS-API-6-CCP_DS_API_MESSAGE: SolandraDS.getByCampaign query: campaignStatus_106162_s:IN AND de.type:SocialContact AND sc.socialContactStatus
unread) [from:null, limit:51]0000000638: 10.86.141.221: Feb 04 2013 10:23:28.658 -0500: %CCBU_____CCP-DS-API-6-CCP_DS_API_MESSAGE: Stored new cache entry com.cisco.ccbu.ccp.ds.cassandra.api.CampaignQueryCache$CacheEntry@42ec93[campaignId=106162,params=com.cisco.ccbu.ccp.ds.beans.CampaignQueryParams@1181a7d[limit=51,fromIndex=<null>,campaignStatus=IN,baseTimestamp=<null>,timestamp=<null>,filterStatus=,filterTag=<null>,socialContact=<null>,socialContactId=<null>,solrSearchQuery=<null>,includeContactWithContactId=false],lastPublished=Wed Oct 31 20:11:32 EDT 2012,firstPublished=Mon Aug 06 04:34:27 EDT 2012]
0000004023: 10.86.141.221: Feb 04 2013 10:23:28.770 -0500: %CCBU_________CCPAPI-6-CAMPAIGN_RESULTS_FOUND: %[query params=com.cisco.ccbu.ccp.ds.beans.CampaignQueryParams@1cb70bf[limit=51,fromIndex=<null>,campaignStatus=IN,baseTimestamp=<null>,timestamp=<null>,filterStatus=,filterTag=<null>,socialContact=<null>,socialContactId=<null>,solrSearchQuery=<null>,includeContactWithContactId=false]]: Found and returning campaign results for query
0000015538: 10.86.141.221: Feb 04 2013 10:23:28.795 -0500: %CCBU_INFRASTRUCTURE-6-REQUEST_END: %[PARAM_TIME_ELAPSED=155]: Request complete
0000015539: 10.86.141.221: Feb 04 2013 10:23:29.111 -0500: %CCBU_INFRASTRUCTURE-6-REQUEST_START: %[ip.orig=10.86.141.221][method_name=GET][parameter_name={ time=[1359991409065], filterStatus=, postId=[B94F54821000013A000070360A568DDD] }][port.orig=46355][resource_name=/campaign/Test_Camp/count]: Request start
0000015540: 10.86.141.221: Feb 04 2013 10:23:29.123 -0500: %CCBU_INFRASTRUCTURE-6-REQUEST_END: %[PARAM_TIME_ELAPSED=11]: Request complete
From: Cisco Developer Community Forums [mailto:cdicuser@developer.cisco.com]
Sent: Monday, February 04, 2013 6:54 AM
To: cdicuser@developer.cisco.com
Subject: New Message from Koushal Pandit in Cisco SocialMiner - Technical Discussion: Xml data Result
Koushal Pandit has created a new message in the forum "Technical Discussion": -------------------------------------------------------------- So I am trying to populate the database with posts from the feed, the function requires xml data from the feed. So I use the following for the data :
http://<server-name>/ccp-webapp/ccp/campaign/CampaignName/results<http://%3cserver-name%3e/ccp-webapp/ccp/campaign/CampaignName/results>
but it is missing out certain posts/comments. Although all the post data is available in Social Miner but the results page doesn't show them. Is it the correct link for the xml data of the feed? Any reason why it's not showing all the data?
--
To respond to this post, please click the following link: http://developer.cisco.com/web/socialminer/forums/-/message_boards/view_message/11514985 or simply reply to this email.
Subject: RE: New Message from Koushal Pandit in Cisco SocialMiner - Technical Discus
Replied by: Koushal Pandit on 07-02-2013 12:48:09 AM
I am not getting the results I wanted using SocialMiner REST API campaign results call. I am not refereeing to a specific time frame that the results are missing for. I am getting the expected results in SocialMIner UI HOME page for this specific campaign. There only some posts that are missing not all.
I even checked cccapi for the correct API call and still the result REST API is not displaying some of the posts even though the SocialMiner UI home is showing those posts when I search them.
Subject: RE: New Message from Koushal Pandit in Cisco SocialMiner - Technical Discus
Replied by: Tetyana Shmukler on 07-02-2013 12:16:23 PM
Hi,
Thank you for getting back to us.
It is hard to troubleshoot based on the provided information. It would help, if you can provide us with the details on what feed you use (its type, url, pooling interval),, as well as the actual API call (the http request), so we can try to reproduce the behavior you see in house.
With the information above, it would be great to have an example of at least one SocialContact (post) that is missing from the result of your API call.
Please also specify what version of SocialMiner you use.
Thank you
Tanya
From: Cisco Developer Community Forums [mailto:cdicuser@developer.cisco.com]
Sent: Thursday, February 07, 2013 1:48 AM
To: cdicuser@developer.cisco.com
Subject: New Message from Koushal Pandit in Cisco SocialMiner - Technical Discussion: RE: New Message from Koushal Pandit in Cisco SocialMiner - Technical Discus
Koushal Pandit has created a new message in the forum "Technical Discussion": -------------------------------------------------------------- I am not getting the results I wanted using SocialMiner REST API campaign results call. I am not refereeing to a specific time frame that the results are missing for. I am getting the expected results in SocialMIner UI HOME page for this specific campaign. There only some posts that are missing not all.
I even checked cccapi for the correct API call and still the result REST API is not displaying some of the posts even though the SocialMiner UI home is showing those posts when I search them.
--
To respond to this post, please click the following link: http://developer.cisco.com/web/socialminer/forums/-/message_boards/view_message/11649968 or simply reply to this email.
Subject: RE: New Message from Koushal Pandit in Cisco SocialMiner - Technical Discus
Replied by: Koushal Pandit on 08-02-2013 12:51:27 AM
Tetyana Shmukler:Hi,
Thank you for getting back to us.
It is hard to troubleshoot based on the provided information. It would help, if you can provide us with the details on what feed you use (its type, url, pooling interval),, as well as the actual API call (the http request), so we can try to reproduce the behavior you see in house.
With the information above, it would be great to have an example of at least one SocialContact (post) that is missing from the result of your API call.
Please also specify what version of SocialMiner you use.
Thank you
Tanya
Thank you for your reply. So the we are basically testing out SocialMiner and the link we used was :
https://www.facebook.com/PUBsg with polling interval of 5 mins
the API we called was http://<server-name>/ccp-webapp/ccp/campaign/PUB_SG/results?resultsPerPage=200
One example is the first post by others (on fb page) :
"Next Depress
RC boats at Bedok Reservoir Floating Platform! I like!"
This post comes in SocialMiner UI and in the result API page too but the comments on this post only come in the SocialMiner UI and not the result API.
Another post was "UPDATE: Pipe repair works in progress at Upper Paya Lebar Rd (opp AUPE Building twds Bartley Rd). 1 out of 4 lanes closed. Works expected to be completed by this afternoon. Issued 7:38am"
The comments on this post do not show up in result page. There are many more examples as you can see. This error did not occur in Social Miner 8.5. The social miner 9.0 is a fresh installation and not update of social miner 8.5. We thought this could be an issue regarding a user's privacy level on facebook but there were several posts with same user whose comments were visible in some posts and not visible in other posts on the API result page.
Another anomaly that we noticed was that comments that are made via mobile do not appear either.
Thank you again for all the help.