i new python , web scapping , trying write basic script data webpage can accessed after logging in. have looked @ bunch of different examples none fixing issue. have far:
from bs4 import beautifulsoup import urllib, urllib2, cookielib username = 'name' password = 'pass' cj = cookielib.cookiejar() opener = urllib2.build_opener(urllib2.httpcookieprocessor(cj)) login_data = urllib.urlencode({'username' : username, 'password' : password}) opener.open('webpagewithloginform') resp = opener.open('webpageiwanttoaccess') soup = beautifulsoup(resp, 'html.parser') print soup.prettify() as of right when print page prints contents of page if not logged in. think issue has way setting cookies not sure because not understand happening cookie processor , libraries. thank you!
current code:
import requests import sys email = 'usr' password = 'pass' url = 'https://connect.lehigh.edu/app/login' def main(): # start session can have persistant cookies session = requests.session(config={'verbose': sys.stderr}) # form data page sends when logging in login_data = { 'username': email, 'password': password, 'login': 'login', } # authenticate r = session.post(url, data=login_data) # try accessing page requires logged in r = session.get('https://lewisweb.cc.lehigh.edu/prod/bwskfshd.p_crseschddetl') if __name__ == '__main__': main()
you can use requests module.
take @ answer i've linked below.
Comments
Post a Comment