Member Avatar for gunbuster363

I am using BeautifulSoup for my project
and I don't know how the object returned by the Soup is organized,
so I don't know how to access it, except using only print

I can only do this:

link = soup.find(attrs={'class' : re.compile("util2$")})
print link

it print out the whole <a href......> .... </a>
but I only want the address within it
do anybody knows?

  • 4 Contributors
  • 7 Replies
  • 2K Views
  • 10 Hours Discussion Span
  • Latest Post Latest Post by gunbuster363

Dani AI

Generated

If soup.find(...) is returning an <a ...> tag, the object you get back is a BeautifulSoup Tag. Two quick checks tell you what you are holding and how to navigate it: type(obj).__name__ for the kind of object, and obj.name/obj.attrs for the HTML tag name and its attributes. From there, grab the link with tag.get('href') (safe) or tag['href'] (raises KeyError if missing). For the visible text, prefer tag.get_text(strip=True); .string often comes back None when the tag has nested elements.

Here is a compact, defensive pattern you can drop in. It avoids printing the whole tag and shows the structure you likely care about:

import re

link = soup.find('a', attrs={'class': re.compile(r'util2$')})

if link is None:
    raise LookupError('No <a> element with class ending in "util2" found.')

# Inspect the object quickly
print('kind:', type(link).__name__)   # e.g., Tag
print('tag :', link.name)             # e.g., 'a'
print('attrs:', link.attrs)           # dict of all attributes

# Extract what you likely wanted
href = link.get('href')               # None if no href
text = link.get_text(strip=True) if hasattr(link, 'get_text') else ''

print('href:', href)
print('text:', text)

Extra tips:

  • In modern BeautifulSoup, CSS selectors can be simpler: a = soup.select_one('a.util2'); href = a.get('href') if a else None.
  • link.prettify() is great for seeing the local subtree without dumping the full document.
  • If you are on an older BS version without get_text, fall back to ''.join(link.findAll(text=True)).strip().

Recommended Answers

All 7 Replies

Member Avatar for lllllIllIlllI

You can find out how something is make up by using the dir() function.

This lists all of the variables, methods and classes of whatever you put inside the brackets :)

Member Avatar for gunbuster363

I do this:
link = soup.find(attrs={'class' : re.compile("util2$")})
print dir(link)

what appear is:

lamwaiman@cs6201:~/python> python m.py

OMG

Member Avatar for lrh9

I do this:
link = soup.find(attrs={'class' : re.compile("util2$")})
print dir(link)

what appear is:


OMG

You can import the module pprint, and using the function pprint, you can display each function and variable on its own line.

from pprint import pprint

link = soup.find(attrs={'class':re.compile("util2$")})
pprint(dir(link))
Member Avatar for gunbuster363

it can only shows the functions of the object///
not something I want.....

Member Avatar for gunbuster363

problem solved.......
I found out how to Navigate it.......

Member Avatar for pythopian

problem solved.......
I found out how to Navigate it.......

And are you going to share your insights, so that others could benefit from them, or keep them to yourself?

Edited by pythopian because: n/a
Member Avatar for gunbuster363

And are you going to share your insights, so that others could benefit from them, or keep them to yourself?

don't be mad.
Just in the doc of BeautifulSoup....

The attributes of Tags

Tag and NavigableString objects have lots of useful members, most of which are covered in Navigating the Parse Tree and Searching the Parse Tree. However, there's one aspect of Tag objects we'll cover here: the attributes.

SGML tags have attributes:. for instance, each of the <P> tags in the example HTML above has an "id" attribute and an "align" attribute. You can access a tag's attributes by treating the Tag object as though it were a dictionary:

firstPTag, secondPTag = soup.findAll('p')

firstPTag
# u'firstPara'

secondPTag
# u'secondPara'

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.